Systems and methods for providing medical information and for performing a medically-related process using augmented reality technology

ABSTRACT

An apparatus for use in a medical process that involves a particle accelerator, includes: a processing unit configured to obtain treatment plan information, obtain a viewing direction of a user of the apparatus, and process the treatment plan information based on the viewing direction of the user of the apparatus to create a graphical representation of the treatment plan information for presentation to the user of the apparatus; and a screen for displaying the graphical representation.

FIELD

The field of the application relates to medical devices, and moreparticularly, to medical devices for providing medical information andfor performing a medically-related process using augmented realitytechnology.

BACKGROUND

Radiation therapy involves medical procedures that selectively deliverhigh doses of radiation to certain areas inside a human body. Also,particle (e.g., electron, proton, etc.) beam treatment may be used toprovide certain treatments. In either radiation therapy or particle beamtreatment, the patient is first positioned next to the treatmentmachine, and a patient setup procedure is performed to align the patientwith the treatment machine. After the patient has been set up, thetechnician then operates the treatment machine to deliver treatmentenergy towards the patient. Currently, the control for controlling thetreatment machine is located in a separate control room that is awayfrom the treatment room where the treatment machine is located. Soduring or after a treatment procedure, the technician may view variousinformation related to the treatment in a screen located in the controlroom. New devices and methods for presenting medical information totechnician are described herein.

SUMMARY

In accordance with one specific implementation of an embodiment, anapparatus includes a wearable augmented reality device, such as virtualreality glasses or a holographic visor. The apparatus is configured forsensing the three-dimensional (3D) shape of the surrounding space,tracking the position of the device in that space, and projecting imagesin the field of view of the user in real-time, to assist in collisiondetection after patient setup. The apparatus is worn by the operator whois positioning the patient in the treatment room. The apparatus builds a3D model of the patient volume during the setup of patient while thepatient is supported on a patient support. The apparatus then simulatesand/or monitors the treatment-time movements of the treatment unit andthe patient (with couch) and alerts for collisions between the two.Additionally, positions where the treatment unit comes to closeproximity of the patient can be readily detected in the holographicprojection by the operator (and algorithmically), which allows thepatient to be advised of such situations in order to alleviate patient'sconcerns regarding collision with the treatment machine.

In accordance with another specific implementation of an embodiment, anapparatus includes a wearable augmented reality device, such as virtualreality glasses or a holographic visor. The apparatus is configured forsensing the three-dimensional (3D) shape of the surrounding space,tracking the position of the device in that space, and projecting imagesin the field of view of the user in real-time, to assist in patientpositioning. The apparatus is worn by the operator who is positioningthe patient in the treatment room. The apparatus builds a 3D model ofthe patient during the patient setup on the support device. Theapparatus then projects the 3D CT/MRI image of the patient as used intreatment planning onto the translucent glass/visor surface within theoperator's field of view. Other visual cues include, but are not limitedto, the position of the isocenter, calculated dose, the treatmentfields, and relevant organs, such as bones and the target(s), or anycombination thereof, on or inside the actual patient as seen by theoperator. The expected positions of patient anatomical features can beprojected as well. The apparatus provides visual cues that help theoperator to align the patient with the expected position and thetreatment machine in all six degrees of freedom, which reducesinter-fraction variance of dose distribution due to patient position.

In accordance with another specific implementation of an embodiment, anapparatus includes a wearable augmented reality (AR) device, such asvirtual reality glasses or a holographic visor. The apparatus isconfigured for projecting three-dimensional images in the field of viewof the user in real-time, to assist in patient identification usingmultiple methods. The apparatus is worn by the operator whocalls/assists the patient in to the treatment room, and/or who isresponsible for patient setup for treatment. The apparatus projects thepatient identification photograph for side-by-side visual comparison.Additionally, biometric identification algorithms, such as face, iris,hand, ear, and voice recognition, can be used to verify patient identitybased on data collected via the sensors of the AR device. The datacollection can take place during transition from waiting/changing roomto the treatment room or inside the treatment room. The apparatus cansupport the use of bar codes, QR codes, or other visual taggingtechnologies, or hand-held radio frequency identification device (RFID)readers, for patient identification. The identification methods can beconfigurable in order to adapt to the identification practices specificto the clinic. The apparatus can automatically identify the patientusing multiple methods simultaneously, and prompt for approval oradditional verification as needed.

In accordance with another specific implementation of an embodiment, anapparatus includes a wearable augmented reality device, such as virtualreality glasses or a holographic visor. The apparatus is configured forbeing used in the treatment for in situ emergency planning directly on apatient CBCT and fast dose calculation, resulting in better treatmentwith less side effects. The apparatus utilizes augmented reality toallow real-time positioning of the patient and treatment beams, whilethe patient is stationary in the treatment room. The augmented realityprovided by the apparatus allows the treater to see the planned dosedelivery directly on the patient. The treater can then shape the doseand try different treatment configurations. In these cases, carefullycontoured structure sets are not available to a treater when makingtheir plans, but can be inferred directly from the patient. Overlayingthe dose on the patient would clarify impact of the treatment on thepatient. In cases where there are no CT images available for dosecalculation, the depth of the treatment isocenter can be rendered on tothe patient by the apparatus, so the treater can estimate the impact ofthe MUs planned, and can adjust the plan if necessary. In some cases,augmented reality can be used to create a volumetric patient model frompatient surface. The patient surface is scanned from the top of thepatient support device and the approximate patient model is filled withwater electron density matter. This gives an approximation of both thepatient shape and the electron densities within the patient.Accordingly, the apparatus can calculate the approximate 3D dosedistribution in certain depth in treatment isocenter. In the cases whereTOF and multiple cameras are not available, the headset of the apparatuscan be used to capture the patient's physique in a similar manner to TOFtechnologies. Using augmented reality, the apparatus can provide thequality of the model to the user, who can continue to take additionalsample points until the model closely relates to the patient.Additionally, in some cases, the apparatus may be configured to modifythis model to match the patient's anatomy (e.g., through deformationtransformation). In cases where CT/CBCT images are available, these canbe rendered by the apparatus in a cross section on the patient, locatedat the treatment isocenter, and orthogonal to the treater. Calculateddose can be rendered by the apparatus on the cross section, thusallowing the treater to view the dose delivery's effects on thepatient's organs. There are several ways of rendering image information,such as: (1) Render images within the patient along the axes of theisocenter lasers, so that the treater can position the patientcorrectly, (2) Render images within the patient orthogonal to thetreater's headset (e.g., by moving the user's head wearing the apparatusto “scan” the patient, a 3-D model of the patient's internal structurecan be estimated). In cases where images are not available, theestimated dose can be rendered in the same way.

In accordance with another specific implementation of an embodiment, anapparatus includes a wearable augmented reality device, such as virtualreality glasses or a holographic visor. The apparatus is configured forbeing used in the treatment room for assisting a user to adjust thepatient to match a treatment plan, and/or for assisting the user todetermine a treatment plan. The apparatus is advantageous over theapproach in which the treatment console and imaging guidance system arein a separate room different from the treatment room in which thepatient is located. The apparatus makes it easier for the user to adjustthe patient to match the treatment plan, without constantly going backand forth between the treatment room to position the patient, and thetreatment console to check the positioning. The apparatus is alsoadvantageous over the approach in which treatment target information isprovided on an iPad. This is because the apparatus may be configured toprovide the same treatment target information, but in an overlayconfiguration directly over the patient as viewed through the screen ofthe apparatus, so that the treatment target appears “inside” thepatient's body. Also, in some embodiments in which the apparatus isconfigured for worn at the user's head, the user may use his/her handsto manipulate the patient, without having to hold the iPad. Thus, theapparatus is an improvement in the technology of patient setup,treatment planning, and treatment execution.

In some cases, the apparatus may provide treatment information (e.g.,planned/accumulated dose, differences in targets and organs shapes andsizes, isocenter locations) for display on the screen of the apparatus,so that the user of the apparatus can adapt the treatment plan while thepatient is in the treatment room. Also, in some cases, isocenter lasersmay be extrapolated within the patient's body (i.e., as viewed by theuser of the apparatus) to help the user of the apparatus align thepatient with the treatment target. Augmented reality provided by theapparatus adds value, because the patient will be simultaneously visibleas the apparatus renders one or more useful features over the patient'sbody as viewed through the screen of the apparatus. Augmented realityprovided by the apparatus may also allow the user of the apparatus toview the patient's treatment location from different angles, beaminformation, and other relevant patient treatment information.

Also, in some cases, the apparatus may display information to inform theuser of changes to the patient's posture, and/or changes to thepatient's anatomy. Such information may assist the user in adjusting thepatient to adapt a treatment plan and/or to determine a new treatmentplan based on the existing condition of the patient. In oneimplementation, one or more modalities may be used to provide detailedinformation about the patient's internal and/or external structure(s),and such information may be displayed on the screen of the apparatus. Insome embodiments, images of structures used for treatment planning maybe displayed together with the current images of the same structures atthe screen of the apparatus. This allows changes of structures to beeasily observed, and can help guide adaptation of the treatment plan.

In some cases, for treatment plan adaptation, the apparatus may providedose variations overlayed on the patient's tumor and critical organs, sothat the user of the apparatus may adjust the patient for the treatmentplan. For example, if a patient's hand is incorrectly placed, this mayimpact a treatment beam, leading to an incorrect dose delivery. Also, insome cases, the apparatus may provide a preview of beam angles andpositions for display on the screen, so that the user of the apparatusmay readily see how the beams will traverse different parts of thepatient. For example, if the user sees that a preview of a beamundesirably traverses a patient's hand, the user may then adjust thepatient's posture accordingly. In further cases, the apparatus mayprovide other guidance information for assisting the user of theapparatus to alter a setup to allow treatment to occur. For example,changes to the patient's anatomy (for example, due to weight loss) wouldalso be evident by comparing planning structure sets with the physicalreality of the patient. In some embodiments, the apparatus may allow theuser to determine a new treatment plan by selecting one of a pluralityof pre-determined treatment plans, or by changing a parameter of acurrent treatment plan.

In other embodiments, instead of providing the various information onthe screen of the apparatus that is for worn at the user's head, thesame information may be projected onto the patient using one or moreprojectors inside the treatment room. In further embodiments, virtualreality technology may be used to instruct the patient to alter his/hersetup (e.g., position, posture, etc.) remotely.

An apparatus for use in a medical process that involves a particleaccelerator, includes: a processing unit configured to obtain medicalinformation, obtain a viewing direction of a user of the apparatus, andprocess the medical information based on the viewing direction of theuser of the apparatus to create a graphical representation of themedical information for presentation to the user of the apparatus; and ascreen for displaying the graphical representation.

Optionally, the apparatus further includes a wearable device, whereinthe screen is a part of the wearable device.

Optionally, the apparatus further includes an orientation sensor coupledto the wearable device, wherein the processing unit is configured tovary the graphical representation based on an input from the orientationsensor.

Optionally, the apparatus further includes a positioning device coupledto the wearable device, wherein the processing unit is configured tovary the graphical representation based on an input from the positioningdevice.

Optionally, the wearable device comprises a virtual-reality device.

Optionally, the screen comprises a transparent screen for allowing theuser to see surrounding space.

Optionally, the screen is a part of a handheld device.

Optionally, the graphical representation has a variable configurationthat corresponds with the viewing direction of the user.

Optionally, the processing unit is also configured to obtain patientinformation regarding a geometry of a patient, wherein the processingunit is configured to process the medical information based on thepatient information and the viewing direction of the user of theapparatus.

Optionally, the apparatus further includes a time-of-flight camera forproviding distance information, wherein the patient informationcomprises a surface of the patient that is based on the distanceinformation.

Optionally, the patient information comprises a digital image of thepatient, a digital image of another person different from the patient,or a model of an artificial patient.

Optionally, the medical information comprises planned dose, delivereddose, image of internal tissue of a patient, target shape, targetposition, critical organ shape, critical organ position, or anycombination of the foregoing.

Optionally, the medical information comprises dose information, andwherein the processing unit is configured to create the graphicalrepresentation of the dose information based on the viewing direction ofthe user, and to provide the graphical representation for display over apatient or for display in an overlay configuration with an image of thepatient.

Optionally, the medical information comprises tissue geometry, andwherein the processing unit is configured to create the graphicalrepresentation of the tissue geometry based on the viewing direction ofthe user, and to provide the graphical representation for display over apatient or for display in an overlay configuration with an image of thepatient.

Optionally, the processing unit is configured to create the graphicalrepresentation along isocenter axes as viewed by the user.

Optionally, the processing unit is also configured to provide a userinterface for allowing the user to determine a treatment parameter for atreatment plan while a patient is supported on a patient support.

Optionally, the processing unit is also configured to obtain patientinformation, the patient information comprising an image of a patient,the medical information comprising dose information, and wherein theprocessing unit is configured to obtain the medical information bycalculating the dose information based on the image of the patient.

Optionally, the image of the patient comprises a CT image.

Optionally, the processing unit is also configured to obtain a patientmodel created based on a detected surface of the patient, wherein theprocessing unit is configured to process the medical information basedon the patient model and the viewing direction of the user of theapparatus to create the graphical representation.

Optionally, the patient model comprises a volumetric model approximatinga shape of the patient and densities within the patient.

Optionally, the medical information comprises dose information, andwherein the processing unit is configured to determine the doseinformation based on the patient model.

Optionally, the medical information comprises a depth of a treatmentisocenter, and the processing unit is also configured to render thedepth of the treatment isocenter over a patient or for display in anoverlay configuration with an image of the patient.

Optionally, the processing unit is also configured to obtain patientinformation, the patient information comprising a position of a patient,and wherein the medical information comprises image data of the patient;and wherein the processing unit is configured to create the graphicalrepresentation of the image data based on the viewing direction of theuser and the position of the patient.

Optionally, the graphical representation comprises a cross section of aCT image.

Optionally, the medical information further comprises dose information,and the graphical representation illustrates the dose information on thecross section of the CT image.

Optionally, the processing unit is configured to create the crosssection of the CT image along isocenter axes.

Optionally, the processing unit is configured to create the crosssection of the CT image along a direction that is orthogonal to theviewing direction of the user.

Optionally, the screen comprises a transparent portion for allowing theuser to view a real world.

Optionally, the screen is a part of a holographic device configured toproject three-dimensional images in a field of view of the user inreal-time.

Optionally, the processing unit is also configured to provide aphotograph of a patient for display on the screen.

Optionally, the apparatus further includes a sensor configured to sensea characteristic of a patient for biometric identification.

Optionally, the characteristic comprises a facial feature, an irisfeature, a retina feature, a hand feature, an ear feature, afingerprint, or a voice.

Optionally, the processing unit is configured to compare the sensedcharacteristic with a pre-determined characteristic of the patient.

Optionally, the apparatus further includes a sensor configured to sensean identification of a patient, wherein the screen is configured todisplay the identification of the patient.

Optionally, the identification comprises a barcode, a quick-response(QR) code, or a RFID.

Optionally, the processing unit is further configured to obtain roominformation, and to generate positional information based on the roominformation for assisting the user to position a patient, and whereinthe processing unit is configured to provide the positional informationfor display on the screen.

Optionally, the room information comprises a position of an object in aroom, the object being a component of a machine, a patient support, awall, a floor, a ceiling, or an alignment device.

Optionally, the medical information comprises an expected position of apatient, and wherein the processing unit is configured to provide thegraphical representation of the expected position of the patient fordisplay on the screen.

Optionally, the screen is configured to display the graphicalrepresentation of the expected position of the patient in a field ofview of the user while the user is viewing the patient in real-time.

Optionally, the apparatus further includes a user interface for allowingthe user to position the patient based on the graphical representationof the expected position of the patient.

Optionally, the apparatus further includes a sensor for sensing anobject next to a patient, wherein the processing unit is configured togenerate a signal for notifying the user in response to the sensedobject being within a certain distance from a surface of the patient.

Optionally, the medical information comprises a safety zone that isabove a surface of the patient.

Optionally, the sensor comprises a surface detector.

Optionally, the medical information comprises an image of the patient.

Optionally, the processing unit is also configured to obtain objectinformation regarding an object involved in the medical process, andprovide the object information for display on the screen to assist invalidation of the object.

Optionally, the object comprises a treatment machine, a patient support,a fixation device for fixing a portion of a patient in place, a bolus, amedication, or an accessory.

A method performed by an apparatus in a medical process that involves aparticle accelerator, comprising: obtaining, by a processing unit of theapparatus, medical information; obtaining, by the processing unit of theapparatus, a viewing direction of a user of the apparatus; processing,by the processing unit of the apparatus, the medical information basedon the viewing direction of the user of the apparatus to create agraphical representation of the medical information for presentation tothe user; and displaying the graphical representation in a screen of theapparatus.

An apparatus for use in a medical process that involves a particleaccelerator, includes: a processing unit configured to obtain treatmentplan information, obtain a viewing direction of a user of the apparatus,and process the treatment plan information based on the viewingdirection of the user of the apparatus to create a graphicalrepresentation of the treatment plan information for presentation to theuser of the apparatus; and a screen for displaying the graphicalrepresentation.

Optionally, the treatment plan information comprises a position of anenergy source for delivering a treatment beam.

Optionally, the graphical representation comprises a line representing atrajectory of the treatment beam.

Optionally, the treatment plan information comprises an expectedconfiguration of a component of a treatment machine.

Optionally, the expected configuration comprises an expected position ofthe component of the treatment machine.

Optionally, the treatment plan information comprises an expected dosefor an internal target of a patient.

Optionally, the graphical representation indicates the expected dosegraphically, and wherein the processing unit is configured to providethe graphical representation for display over a patient as viewedthrough the display, or for display in an overlay configuration with animage of the patient, so that the graphical representation is at alocation that corresponds with a position of the internal target of thepatient.

Optionally, the treatment plan information comprises an expected postureof a patient.

Optionally, the treatment plan information comprises a target position,a target size, a target shape, a critical organ position, a criticalorgan size, a critical organ shape, or any combination of the foregoing.

Optionally, the treatment plan information comprises a target fluence,and the processing unit is configured to provide the graphicalrepresentation for representing the target fluence.

Optionally, the treatment plan information comprises a trajectory of acomponent of a treatment machine, and wherein the graphicalrepresentation is configured to indicate the trajectory of the componentof the treatment machine.

Optionally, the processing unit is configured to simulate a treatmentbased on the treatment plan information, and wherein the graphicalrepresentation comprises one or more images represented the simulatedtreatment.

Optionally, the one or more images comprises a sequence of imagesforming a video.

Optionally, each of the images in the video is based on a viewingdirection and position of the user of the apparatus.

Optionally, the simulated treatment comprises a simulated movement of acomponent of a treatment machine.

Optionally, the graphical representation comprises a video showing thesimulated movement of the component of the treatment machine.

Optionally, the apparatus further includes a user interface for allowingthe user to determine a new treatment plan by selecting the newtreatment plan from a plurality of pre-determined treatment plans, whilea patient is supported on a patient support.

Optionally, the apparatus further includes a user interface for allowingthe user to determine a new treatment plan by changing a parameter of acurrent treatment plan, while a patient is supported on a patientsupport.

Optionally, the apparatus further includes a wearable device, whereinthe screen is a part of the wearable device.

Optionally, the apparatus further includes an orientation sensor coupledto the wearable device, wherein the processing unit is configured tovary the graphical representation based on an input from the orientationsensor.

Optionally, the apparatus further includes a positioning device coupledto the wearable device, wherein the processing unit is configured tovary the graphical representation based on an input from the positioningdevice.

Optionally, the wearable device comprises a virtual-reality device.

Optionally, the screen comprises a transparent screen for allowing theuser to see surrounding space.

Optionally, the screen is a part of a handheld device.

Optionally, the graphical representation has a variable configurationthat corresponds with the viewing direction of the user.

Optionally, the apparatus further includes a camera unit coupled to theprocessing unit.

Optionally, the camera unit comprises an optical camera, a depth camera,or both the optical camera and the depth camera.

Optionally, the processing unit is configured to provide the graphicalrepresentation for display over a patient as viewed through the display,or for display in an overlay configuration with an image of the patient.

Optionally, the processing unit is also configured to determine an imageof a patient, and output the image of the patient for display on thescreen based on the viewing direction of the user of the apparatus.

Optionally, the image comprises a CT image, a x-ray image, a MRI image,an ultrasound image, a tomosynthesis image, an on-line image, or a doseimage.

Optionally, the processing unit is configured to determine a treatmentdose, and output a graphic representing the treatment dose for displayon the screen based on the viewing direction of the user of theapparatus.

Optionally, the apparatus further includes a user interface for allowingthe user to control a position of an energy source, a patient support,one or more camera(s), one or more alignment laser(s), one or morelight(s), a calibration of a device, a speaker for communication with apatient, music for presentation to the patient, or any combination ofthe foregoing.

Optionally, the processing unit is configured to receive a real-timeconsultation from a person who is different from the user of theapparatus, and provide guidance information for display on the screenfor assisting the user to determine and/or to adapt a treatment plan.

Optionally, the processing unit is configured to obtain multiplepositions of an isocenter at different respective times, and provide agraphic indicating change(s) of the isocenter over time for display onthe screen.

Optionally, the processing unit is configured to obtain multiple valuesof dose at different respective times, and provide a graphic indicatinghow the dose varies over time.

Optionally, the processing unit is configured to provide patient setupinformation for display on the screen, the patient setup informationindicating weight change and/or positional change, of a patient.

Optionally, the processing unit is configured to provide informationregarding fluence virtualization for display on the screen.

Optionally, the apparatus further includes a database configured tostore data documenting one or more activities that occur in a treatmentroom.

Optionally, the data represents a treatment setup configuration, apatient setup configuration, a patient behavior, or any combination ofthe foregoing.

Optionally, the data indicates how a treatment was executed.

Optionally, the data indicates positions of a component of a treatmentmachine at different respective times, and/or a timing of energydelivery.

Optionally, the screen comprises a transparent portion for allowing theuser to view a real world.

Optionally, the screen is a part of a holographic device configured toproject three-dimensional images in a field of view of the user inreal-time.

Optionally, the treatment plan information comprises a simulated doseeffect on a target region and/or critical organ.

Optionally, the processing unit is configured to simulate an executionof a treatment plan to determine the simulated dose effect on the targetregion and/or critical organ.

A method performed by an apparatus in a medical process that involves aparticle accelerator, includes: obtaining, by a processing unit of theapparatus, treatment plan information; obtaining, by the processing unitof the apparatus, a viewing direction of a user of the apparatus;processing, by the processing unit of the apparatus, the treatment planinformation based on the viewing direction of the user of the apparatusto create a graphical representation of the treatment plan informationfor presentation to the user; and displaying the graphicalrepresentation in a screen of the apparatus.

Other and further aspects and features will be evident from reading thefollowing detailed description.

DESCRIPTION OF THE DRAWINGS

The drawings illustrate the design and utility of embodiments, in whichsimilar elements are referred to by common reference numerals. Thesedrawings are not necessarily drawn to scale. In order to betterappreciate how the above-recited and other advantages and objects areobtained, a more particular description of the embodiments will berendered, which are illustrated in the accompanying drawings. Thesedrawings depict only exemplary embodiments and are not therefore to beconsidered limiting in the scope of the claims.

FIG. 1 illustrates an apparatus for use in a medical process.

FIG. 2A illustrates a treatment system with which the apparatus of FIG.1 can be used.

FIG. 2B illustrates another treatment system with which the apparatus ofFIG. 1 can be used.

FIG. 3A-3B illustrate an example of the apparatus providing a graphicalrepresentation of medical information in an overlay configuration withrespect to a patient or an image of the patient.

FIG. 3C illustrates what the user will see without the benefit of theapparatus of FIG. 3A.

FIGS. 4A-4B illustrates another example of the apparatus providing agraphical representation of medical information in an overlayconfiguration with respect to a patient or an image of the patient.

FIG. 4C illustrates what the user will see without the benefit of theapparatus of FIG. 4A.

FIG. 5 illustrates information flow for the apparatus of FIG. 1.

FIG. 6 illustrates a treatment system having a camera system that may beused with the apparatus of FIG. 1.

FIG. 7 illustrates a method in accordance with some embodiments.

FIG. 8 illustrates another method in accordance with some embodiments.

FIG. 9 illustrates a specialized processing system.

DETAILED DESCRIPTION

Various embodiments are described hereinafter with reference to thefigures. It should be noted that the figures are not drawn to scale andthat elements of similar structures or functions are represented by likereference numerals throughout the figures. It should also be noted thatthe figures are only intended to facilitate the description of theembodiments. They are not intended as an exhaustive description of theinvention or as a limitation on the scope of the invention. In addition,an illustrated embodiment needs not have all the aspects or advantagesshown. An aspect or an advantage described in conjunction with aparticular embodiment is not necessarily limited to that embodiment andcan be practiced in any other embodiments even if not so illustrated, orif not so explicitly described.

FIG. 1 illustrates an apparatus 10 for use in a medical process. Theapparatus 10 includes a processing unit 12 and a screen 14 configuredfor displaying a graphical representation of medical information for auser of the apparatus 10. The processing unit 12 is configured to obtainmedical information, obtain a viewing direction of the user of theapparatus, and process the medical information based on the viewingdirection of the user of the apparatus 10 to create the graphicalrepresentation of the medical information for presentation to the userof the apparatus 10.

As shown in the figure, the processing unit 12 of the apparatus 10includes a medical information module 20 configured to obtain medicalinformation, a patient information module 22 configured to obtainpatient information, and a viewing direction module 24 configured toobtain a viewing direction of the user of the apparatus 10. Theprocessing unit 12 also includes a graphics generator 30 coupled to themedical information module 20, the patient information module 22, andthe viewing direction module. The graphics generator 30 is configured toreceive the medical information from the medical information module 20,receive the patient information from the patient information module 22,and the viewing direction from the viewing direction module 24, andcreate the graphical representation of the medical information fordisplay on the screen 14 of the apparatus 10 for viewing by the user ofthe apparatus 10.

In the illustrated embodiments, the processing unit 12 also optionallyincludes a room information module 32 configured to obtain roominformation. In some cases, the processing unit 12 may create thegraphical representation of the medical information also based on theroom information from the room information module 32.

The processing unit 12 may also optionally include a user interface 34configured to receive user input from the user of the apparatus 10. Theuser interface 34 may be configured to allow a user to enter a command,such as a selection of the type of medical information for display onthe screen 14, the format of the graphical representation of the medicalinformation, etc. The user interface 34 may also be configured toreceive input from the user for controlling a medical device, such as atreatment planning device, a treatment device, an imaging device, apatient support, or any combination of the foregoing.

The processing unit 12 may also optionally include a non-transitorymedium 36 for storing data. The data may be medical information obtainedby the medical information module 20, patient information obtained bythe patient information module 22, viewing direction obtained by theviewing direction module 24, room information obtained by the roominformation module 32, or any combination of the foregoing. Also, thedata stored in the non-transitory medium may be information derived fromthe patient information, from the room information, from the viewingdirection, or any combination of the foregoing. In some embodiments, thenon-transitory medium 36 may also store a treatment plan for aparticular patient, and patient identity information for a particularpatient.

As shown in FIG. 1, the apparatus 10 is in a form of a wearable devicethat includes the screen 14, and a frame 60 to which the screen 14 issecured. In some embodiments, the screen 14 may be transparent (e.g., atleast partially transparent) for allowing the user of the apparatus 10to see the real world (e.g., surrounding environment). The screen 14 maybe configured to display the graphics from the graphics generator 30 sothat the graphics are superimposed with real objects as directly viewedby the user. Alternatively, the wearable device may be a virtual-realitydevice. In such cases, the screen 14 is not transparent, and isconfigured to provide electronic images for viewing by the user. Theimages may represent the environment around the user, and may bedisplayed in real-time. Accordingly, the images presented by theelectronic screen 14 may change in real time in accordance with aviewing direction of the user.

In other embodiments, the screen 14 may be a part of a holographicdevice configured to project three-dimensional images in a field of viewof the user in real-time.

In further embodiments, the screen 14 may be a part of a handhelddevice. By means of non-limiting examples, the handheld device may be acell phone (e.g., an IPHONE), an IPAD, an IPAD MINI, a tablet, etc.

In some embodiments, the apparatus 10 includes an orientation sensorcoupled to the wearable device. For example, the orientation sensor mayinclude one or more accelerometer(s). In such cases, the processing unit12 may be configured to vary the graphical representation displayed onthe screen 14 based on an input from the orientation sensor. Forexample, as the user of the apparatus 10 tilts or turns his/her head,the processing unit 12 will correspondingly vary the graphics on thescreen 14 to match the viewing orientation of the user. Also, in someembodiments, the apparatus 10 includes a positioning device coupled tothe wearable device. The positioning device is configured to determine aposition of the apparatus 10 with respect to some defined coordinate.The positioning device may use active signals or passive signals togenerate positional information regarding a position of the apparatus10. The processing unit 12 is configured to vary the graphicalrepresentation displayed on the screen 14 based on an input from thepositioning device. For example, if a user moves further away from thepatient, the processing unit 12 will correspondingly vary the graphics(e.g., reduce the size of the graphics) on the screen 14 to match theviewing distance. In further embodiments, the apparatus 10 may includeboth an orientation sensor and a positioning device. In such cases, thegraphical representation displayed on the screen 14 has a variableconfiguration that corresponds with the viewing direction and viewingdistance of the user.

In some embodiments, in addition to the medical information, theprocessing unit 12 is configured to obtain patient information regardinga geometry of a patient. In such cases, the processing unit 12 may beconfigured to process the medical information based on both (1) thepatient information and (2) the viewing direction of the user of theapparatus 10. By means of non-limiting examples, the patient informationmay be an image of a person (such as, a digital image of the patient, adigital image of another person different from the patient, or a modelof an artificial patient), a size of the patient, a shape of thepatient, etc. In some cases, the processing unit 12 may be configured togenerate a graphics based on the medical information, and transmit thegraphics for display on the screen 14 in a superimposed configurationwith respect to the image of the person. In other cases, the patientinformation may be information regarding a geometry of the patient, andthe processing unit 12 may be configured to generate the graphicsrepresenting the medical information based on the patient geometry. Inone implementation, patient information may be obtained using one ormore camera(s). The camera(s) may be optical camera(s), and/ortime-of-flight camera(s) configured to provide distance information. Thecamera(s) may be attached or implemented at the apparatus 10.Alternatively, the camera(s) may be secured to another object (e.g., awall, a ceiling, a floor, a patient support, a part of a treatmentdevice, etc.) located in a treatment room. In further embodiments, acamera may be attached or implemented at the apparatus 10, while anothercamera may be secured to another object in the treatment room. In theembodiment in which the camera is a time-of-flight camera, the cameramay provide information regarding a surface of the patient that is basedon the distance information. In such cases, the output from the cameramay be used by the processing unit 12 to generate the surface of thepatient, or a model representing a surface of the patient.

In other embodiments, the patient information itself may be consideredas an example of medical information.

In further embodiments, the medical information may comprise planneddose, delivered dose, image of internal tissue of a patient, targetshape, target position, critical organ shape, critical organ position,or any combination of the foregoing. The processing unit 12 isconfigured to provide a graphics representing such medical informationfor display on the screen 14, so that the graphics appears in an overlayconfiguration with respect to the patient, or with respect to an image(e.g., a real-time image) of the patient.

In some embodiments in which the medical information comprises doseinformation, the processing unit 12 may be configured to create thegraphical representation of the dose information based on the viewingdirection of the user, and to provide the graphical representation fordisplay over a patient or for display in an overlay configuration withan image of the patient.

Also, in some embodiments, the medical information may comprise tissuegeometry (e.g., tissue size, shape, etc.). In such cases, the processingunit 12 may be configured to create the graphical representation of thetissue geometry based on the viewing direction of the user, and toprovide the graphical representation for display over a patient or fordisplay in an overlay configuration with an image (e.g., a real-timeimage) of the patient.

In one or more of the embodiments described herein, the processing unit12 may be configured to create the graphical representation of themedical information along one or more isocenter axes as viewed by theuser. Alternatively, the processing unit 12 may be configured to createthe graphical representation of the medical information along adirection that is orthogonal to the viewing direction of the user of theapparatus 10. In further embodiments, the orientation of the graphicsrepresenting the medical information may be user-prescribed. In oneimplementation, the apparatus 10 may include a user interface (e.g.,with one or more buttons and/or controls) for allowing the user of theapparatus 10 to select a direction of the cross section of an organ ortissue for display on the screen 14 in an overlay configuration withrespect to the patient or with respect to an image (e.g., real-timeimage) of the patient. For example, if the user wants to see a certaincross section of the liver of the patient while the patient is supportedon the patient support, the user may use the user interface of theapparatus 10 to prescribe such cross section with the desiredorientation. In such cases, the processing unit 12 will process the userinput and derive the cross section based on a CT image of the patient.In some embodiments, the user interface of the apparatus 10 may alsoallow the user to select which organ or tissue to display on the screen14.

In other embodiments, the user interface may also allow the user of theapparatus 10 to determine a treatment parameter for a treatment planwhile a patient is supported on a patient support. By means ofnon-limiting examples, the treatment parameter may be a target positionto which treatment energy is to be delivered, a critical organ positionat which treatment energy is to be limited or avoided, a collision-freezone for protecting the patient (i.e., components of the treatmentsystem cannot move within such collision-free zone), etc.

In addition, in some embodiments, the processing unit 12 may beconfigured to obtain a CT image of a patient as an example of patientinformation, and the medical information may be dose information. Insuch cases, the processing unit 12 may be configured to obtain themedical information by calculating the dose information based on the CTimage. For example, one or more anatomical features obtained from the CTimage may be utilized in the determination of dose information. Theprocessing unit 12 then generates a graphics representing the doseinformation for display on the screen 14 of the apparatus 10.

In further embodiments, the processing unit 12 may be configured toobtain a patient model created based on a detected surface of thepatient. The detected surface may be obtained using output from one ormore time-of-flight cameras (e.g., depth cameras). In such cases, theprocessing unit 12 may be configured to process the medical informationbased on the patient model and the viewing direction of the user of theapparatus 10 to create the graphical representation for display on thescreen 14 of the apparatus 10. In some cases, the patient model maycomprise a volumetric model approximating a shape of the patient anddensities within the patient. In one specific example, the patient modelmay be a CT image, or a cross section of a CT image.

In further embodiments, the medical information may comprise doseinformation. In such cases, the processing unit 12 may be configured todetermine the dose information based on the patient model. For example,the patient model may be used by the process unit 12 to determinecertain fiducial point(s) of the patient. The fiducial point(s)establishes certain position and orientation of the patient. Based onthe position and orientation of the patient, the processing unit 12 maythen create a graphics representing dose information so that the doseinformation will be aligned with the correct part of the patient (or thecorrect part of the image of the patient) when the dose information isdisplayed on the screen 14.

In other embodiments, the medical information may comprise a depth of atreatment isocenter. In such cases, the processing unit 12 may beconfigured to render the depth of the treatment isocenter over a patient(e.g., with respect to a viewing direction of the user of the apparatus10), or for display in an overlay configuration with an image (e.g., areal-time image) of the patient.

In some embodiments, the processing unit 12 may also be configured toobtain patient information. For example, the patient information maycomprise a position of a patient. Also, the processing unit 12 mayobtain image data of the patient as another example of the medicalinformation. In such cases, the processing unit 12 may be configured tocreate the graphical representation of the image data based on theviewing direction of the user and the position of the patient. The imagedata may be CT image, ultrasound image, PET image, SPECT image, PET-CTimage, MRI image, x-ray image, etc. In some embodiments, if the imagedata is a CT image, the graphical representation provided by theprocessing unit 12 may comprise a cross section of a CT image. In oneimplementation, the processing unit 12 may be configured to create thecross section of the CT image along isocenter axes. Alternatively, theprocessing unit 12 may be configured to create the cross section of theCT image along a direction that is orthogonal to the viewing directionof the user of the apparatus 10. In some cases, the medical informationmay also comprise dose information. In such cases, the graphicalrepresentation provided by the processing unit 12 may illustrate thedose information on the cross section of the CT image.

Patient Identification

In some embodiments, the processing unit 12 may be configured to providea photograph of a patient for display on the screen 14. This allows theuser of the apparatus 10 to verify an identity of the patient bycomparing the photograph as it appears on the screen 14 and the patientas directly viewed by the user (if the screen is transparent).

In some embodiments, the apparatus 10 may optionally further include asensor configured to sense a characteristic of a patient for biometricidentification. By means of non-limiting examples, the characteristicmay be a facial feature, an iris feature, a retina feature, a handfeature, an ear feature, a fingerprint, a voice, etc. Accordingly, thesensor may be a facial feature detector, an iris feature detector, aretina feature detector, a hand feature detector, an ear featuredetector, a fingerprint detector, a microphone, etc. In oneimplementation, the sensor may be implemented using one or morecamera(s). The sensor may be fixedly attached to, or implemented at, theapparatus 10. Alternatively, the sensor may be communicatively coupledto a component of the apparatus 10. For example, the sensor may becommunicatively coupled to the frame 60 via a cable or via a wirelesstransceiver. The processing unit 12 is configured to receive the sensedcharacteristic from the sensor, and may include a comparator configuredto compare the sensed characteristic with a pre-determinedcharacteristic of the patient. If the sensed characteristic matches withthe pre-determined characteristic, then the processing unit 12 maygenerate a signal to inform the user of the apparatus 10 that theidentity of the patient is confirmed. The signal may be an audio signal,a visual signal, or both. On the other hand, if the sensedcharacteristic does not match with the pre-determined characteristic,then the processing unit 12 may generate a signal to inform the user ofthe apparatus 10 that the identity of the patient is not confirmed. Suchsignal may be an audio signal, a visual signal, or both.

In further embodiments, the apparatus 10 may also optionally include anidentification sensor configured to sense an identification of apatient. By means of non-limiting examples, the identification sensormay be a barcode sensor configured to sense (e.g., read) a barcode, aquick-response (QR) sensor configured to obtain a QR response, a RFIDsensor configured to sense an ID using radiofrequency, etc. Theidentification sensor may be fixedly attached to, or implemented at, theapparatus 10. Alternatively, the identification sensor may becommunicatively coupled to a component of the apparatus 10. For example,the identification sensor may be communicatively coupled to the frame 60via a cable or via a wireless transceiver. The processing unit 12 mayprocessed the sensed identification and output it for display on thescreen 14. The processing unit 12 may also be configured to receive thesensed identification from the identification sensor, and may include acomparator configured to compare the sensed identification with apre-determined identification of the patient. If the sensedidentification matches with the pre-determined identification, then theprocessing unit 12 may generate a signal to inform the user of theapparatus 10 that the identification of the patient is confirmed. Thesignal may be an audio signal, a visual signal, or both. On the otherhand, if the sensed identification does not match with thepre-determined identification, then the processing unit 12 may generatea signal to inform the user of the apparatus 10 that the identificationof the patient is not confirmed. Such signal may be an audio signal, avisual signal, or both.

It should be noted that the apparatus 10 is not limited to using theabove patient information for identifying the patient or for assistingthe identification of the patient. In other embodiments, the apparatus10 may use other types of patient information. For examples, in otherembodiments, the processing unit 12 may provide information regarding anage of the patient, a diagnose of the patient, a treatment site for thepatient, name, identification, etc. for display on the screen 14 of theapparatus 10. The user of the apparatus 10 may utilize such patientinformation to confirm that the patient on the patient support is theintended patient.

In further embodiments, the apparatus may include a microphone forreceiving a sound from the patient. The processing unit 12 may performvoice recognition (e.g., via a voice recognition module) to see if thereceived voice matches with that for the intended patient. If so, theprocessing unit 12 may generate an indicator to inform the user that thepatient identification is correct.

In other embodiments, the apparatus may include an eye-feature detectorfor detecting a feature of the eye of the patient. The processing unit12 may perform eye recognition (e.g., via an eye recognition module) tosee if the detected eye feature matches with that for the intendedpatient. If so, the processing unit 12 may generate an indicator toinform the user that the patient identification is correct.

Other Patient Information

It should be noted that the processing unit 12 is not limited toproviding the above patient information for display on the screen 14 ofthe apparatus 10. The processing unit 12 may also provide other patientinformation for display on the screen 14. By means of non-limitingexamples, the processing unit 12 may provide information to indicatedisease information of the patient, existing pre-conditions of thepatient, future appointment(s) of the patient, warnings (e.g., bloodsample results that may prevent treatment), insurance for the patient,billing status, etc. As another example of patient information, theprocessing unit 12 may also provide patient workflow information—e.g.,treatment planning task, treatment task, imaging task, diagnostic tasks,etc., for the patient. As a further example of patient information, theprocessing unit 12 may also provide questions for the user of theapparatus 10 to ask the patient while the patient is supported on thepatient support.

As other examples of patient information, the processing unit 12 mayalso provide an image (two-dimensional image or a three-dimensionalimage) of a target in the patient for display on the screen 14. In theembodiment in which the screen 14 has a see-through region for allowingthe user to view the patient directly, the image of the target may bedisplayed on the screen 14 so that when the user views the patientthrough the screen 14, the image of the target appears over the patient.Such feature allows the user of the apparatus 10 to perform patientpositioning. In other cases, instead of, or in addition to, image of thetarget, the processing may provide a body outline of the patient, and/orimage(s) (e.g., two-dimensional image(s) or three-dimensional image(s))of target from treatment simulation(s) or from previous treatment(s),for display on the screen 14. These information may also be helpful inassisting the user of the apparatus 10 to perform patient positioning.

In another example of patient information, the processing unit 12 mayprovide an image of virtual tattoo(s) for display on the screen 14. Thevirtual tattoo(s) has predetermined position(s) with respect to thepatient, and may be used by the user of the apparatus 10 to performpatient positioning.

In some cases, to assist the user of the apparatus 10 in performingpatient positioning, the processing unit 12 may also provide virtuallasers for display on the screen 14. In the embodiment in which thescreen 14 has a see-through region for allowing the user to view thepatient directly, the virtual lasers may be displayed on the screen 14so that when the user views the patient through the screen 14, the imageof the virtual lasers will appear over, or extending inside, thepatient.

The processing unit 12 may also provide other positioning aids fordisplay on the screen 14 to assist the user of the apparatus 10 inperforming patient setup. For example, the processing unit 12 mayprovide graphics for allowing the user to visualize a side of thepatient/patient support/fixation device/treatment device that may beobstructed by treatment device or other object(s). As another example,the processing unit 12 may provide image(s) of implant(s) in the patientfor display in the screen 14 so that the implant(s) image(s) will appearover the patient (when the user view the patient directly through thetransparent region of the screen 14) in correspondence with the actualposition(s) of the implant(s). By means of non-limiting examples, theimplant(s) may be radiopaque marker(s), active transmitter(s), passivetransmitter(s), gold seed(s), etc.

As other examples of patient information, the processing unit 12 may beconfigured to obtain information regarding a pacemaker in the patient,electrocardiogram (ECG) for the patient, electromyography (EMG) for thepatient, positioning signals for the patient, or any combination of theforegoing. The processing unit 12 may provide such information fordisplay on the screen 14 of the apparatus. In some cases, thepositioning signals may be signals output from one or more implants,such as Calypso implants. The information regarding the pacemaker may bea position of the pacemaker, and/or signals and timing of signals of thepacemaker. The information regarding ECG may be a position of the ECGdevice, and/or signals and timing of signals of the ECG. The informationregarding the EMG for the patient may be a position of the EMG device,and/or data provided by the EMG device. The information regardingpositioning signals for the patient may be a position of the deviceproviding the positioning signals (e.g., position of an implant), and/orthe positioning signals.

Workflow Assistance Information

Also, in some embodiments, the processing unit 12 may be configured toprovide workflow assistance information for display on the screen 14 toassist the user of the apparatus 10 to perform treatment setup. By meansof non-limiting examples, the workflow assistance information mayinclude tasks checklist, timer for the next treatment, remaining timefor next appointment, etc. Also, in some cases, the workflow assistanceinformation may be identification of object(s) in the treatment roomthat need to be operated on (e.g., for setup).

Room Information

In some embodiments, the processing unit 12 is further configured toobtain room information, and to generate positional information based onthe room information for assisting the user of the apparatus 10 toposition a patient. In such cases, the processing unit 12 may beconfigured to provide the positional information for display on thescreen 14.

In some cases, the room information may comprise a position of an objectin a room. By means of non-limiting examples, the object may be acomponent of a machine, a patient support, a wall, a floor, a ceiling,an alignment device, etc.

Also, in some embodiments, the positional information generated based onthe room information may be a three-dimensional position of the objectin the room with respect to certain coordinate (e.g., a coordinate ofthe apparatus 10). In one implementation, the positional information maybe generated by the processing unit 12 based on a transformation thatconvert the position of the object in the room in a first coordinatesystem to the position in a second coordinate system.

In other embodiments, the positional information may be a desired(expected) position of the patient with respect to the position of theobject in the room. In such cases, the processing unit 12 may beconfigured to determine the desired (expected) position of the patientwith respect to the position of the object in the room, and provide thedesired position as the positional information for display on the screen14.

In further embodiments, the positional information may be an actualposition of the patient with respect to the position of the object inthe room. In such cases, the processing unit 12 may be configured todetermine the actual position of the patient with respect to theposition of the object in the room, and provide the actual position ofthe patient as the positional information for display on the screen 14.

Also, in some cases, the medical information may be an actual positionof a patient. In such cases, the processing unit 12 may be configured toprovide the graphical representation of the actual position of thepatient for display on the screen 14. The processing unit 12 may alsoprovide a graphical representation of the desired position of thepatient for display on the screen 14, so that the user of the apparatus10 can see the difference between the actual position and the desiredposition of the patient. In other embodiments, the medical informationmay be a desired position of the patient.

In some embodiments, the screen 14 is configured to display thegraphical representation of the expected position and/or the actualposition of the patient in a field of view of the user while the user isviewing the patient in real-time. Accordingly, as the user moves aroundto change the field of view, the expected position and/or the actualposition of the patient as displayed on the screen 14 is updated inreal-time.

In some embodiments, the apparatus 10 may further include a userinterface for allowing the user to position the patient based on thegraphical representation of the expected position of the patient. Forexample, based on the actual position of the patient and the expectedposition of the patient, the user interface may be operated by the userto move the patient so that the actual position of the patient isaligned with the expected position of the patient. In oneimplementation, the user interface may control a position of a patientsupport so that movement of the patient may be achieved by movement ofthe patient support supporting the patient.

In other embodiments, the processing unit 12 may be configured to obtainroom information for device validation. For example, the apparatus 10may include a detector (e.g., a camera, a marker detector, an identifierdetector, etc.) for detecting the presence of one or more objects in thetreatment room. The object may be a treatment machine, a patientsupport, a fixation device (e.g., a face mask, a harness, etc.) forfixing a portion of the patient in place, a bolus, a medication, anaccessory, etc. In some cases, the processing unit 12 may be configuredto determine an outline of an object, and perform shape analysis todetermine if the object matches an object that is expected to be usedduring treatment (e.g., an object prescribed in the treatment plan). Ifthe object matches the prescribed object, then the processing unit 12may generate a message for display on the screen 14 to inform the userthat the detected object in the treatment room is validated. Also, insome embodiments, the processing unit 12 may provide virtual objects fordisplay on the screen 14 to inform the user that those objects need tobe provided in the treatment room. The user may then look for thoseobjects to confirm that they are presence and to validate the objects.The virtual objects may be photographs of the objects, three-dimensionalmodels of the objects, three-dimensional photographs of the objects, oridentifiers of the objects.

Collision Avoidance

In some embodiments, the apparatus 10 may also optionally include anobject sensor for sensing a patient, and object(s) next to the patient.In one implementation, the object sensor may be a depth sensing camera,such as a TOF camera. In other embodiments, the object sensor may beother types of sensor, such as a surface detector, etc. The processingunit 12 is configured to receive distance information from the objectsensor, and determine surfaces of objects based on the distanceinformation. The processing unit 12 may also be configured to identifythe objects based on the determined surfaces. Also, in some embodiments,the processing unit 12 may be configured to generate a signal fornotifying the user of the apparatus 10 in response to a sensed objectbeing within a certain distance from a surface of the patient. Forexample, a safety zone may be determined to be 4 inches above thesurface of the patient. In such cases, the processing unit 12 maydetermine a surface model having a surface that is 4 inches offset fromthe surface of the patient. During treatment, the processing unit 12 maybe configured to continuously detect the objects around the patient inreal-time. If an object is detected to be within the safety zone, thenthe processing unit 12 may generate a warning signal (e.g., an audiosignal, a visual signal, or both) to inform the user of the apparatus10. The processing unit 12 may also generate a control signal to stop anoperation of a medical device, such as a treatment device that is beingused to treat the patient.

In some embodiments, the graphics generator 30 may be configured togenerate graphics representing one or more detected objects around thepatient. The object may be a gantry, an energy source, a portal imager,a patient support, a fixation device (e.g., a mask, a harness, etc.) formaintaining a part of the patient stationary with respect to anotherobject, etc. The graphics may be displayed on the screen 14 for viewingby the user of the apparatus 10.

In some embodiments, if the apparatus is a wearable device, the objectsensor may be fixedly attached to the wearable device. In otherembodiments, the object sensor may be fixedly attached to a treatmentmachine, a patient support, an imaging device, or any object in atreatment room (such as a floor, a wall, a ceiling, etc.). In suchcases, the object sensor may be communicatively coupled to the wearabledevice via a cable or a wireless transmitter.

In one implementation, the processing unit 12 may include a collisionprevention module configured to prevent collision between the patientand surrounding object(s), and/or between two or more objects. Forexample, the collision prevention module may be configured to monitordevices surrounding the patient, and generate a warning signal if twoobjects are within a certain prescribed distance. In some cases, thecollision prevention module may be configured to monitor a moving gantryand a patient support to prevent these two devices from colliding. Inother cases, the collision prevention module may be configured tomonitor an imager (e.g., a kV imager) and a positioning device (e.g.,Calypso console) located next to the patient to prevent these twodevices from colliding.

Also, in some embodiments, the processing unit 12 may be configured togenerate a collision-free envelope for display on the screen 14 of theapparatus 10. The collision-free envelope may be generated based onpositions of the various objects detected in the treatment room. Forexample, the apparatus 10 may include a camera, or any of other types ofsensing device, for detecting the treatment machine. The processing unit12 may be configured to determine the position of the treatment machine,as well as the position(s) of one or more components associated with thetreatment machine, such as the position of a moveable radiation source.Based on a degree of freedom and/or designed movement path of theradiation source, the processing unit 12 determines a three dimensionalspace in which the radiation source may be placed. The processing unit12 may determine corresponding three dimensional space for other objectsin the treatment room. By assembling all of these three dimensionalspaces in which positions of the various objects are possible, theprocessing unit 12 can then determine a collision-free space in whichnone of the objects (other than the patient) can be placed. In someembodiments, the processing unit 12 may be configured to providegraphics for display on the screen 14 so that the colliding parts of thevarious items (e.g., patient, patient support, fixation device,treatment machine, etc.) can be visualized by the user of the apparatus10. In addition, in some embodiments, the processing unit 12 may beconfigured to provide a graphic indicating a collision free zone fordisplay on the screen 14 for assisting the user of the apparatus 10 toposition the patient. In some cases, the collision free zone may bedetermined by the processing unit 12 by determining a collision-freespace that the objects around the patient cannot be moved to. Suchcollision-free space may then be used as the collision free zone fordisplay on the screen 14. Alternatively, the processing unit 12 mayprovide a safety of margin by reducing such collision-free space furtherto obtain a reduced collision-free space. The reduced collision-freespace may then be used as the collision free zone for display on thescreen 14.

Furthermore, in some embodiments, the processing unit 12 may beconfigured to perform a virtual dry-run by simulating movements of thevarious objects in the treatment room. The simulated movements may bebased on the actual condition (e.g., position, shape, etc.) of theobjects as detected in the treatment room. For example, the simulatedmovement of the object may be conducted so that the movement path beginsat, or intersects with, the actual position of the object. Theprocessing unit 12 may also provide graphics indicating the simulatedmovements of the objects for display on the screen 14 of the apparatus10. For example, the processing unit 12 may provide a video showing howan object moves relative to the patient. In the embodiment in which thescreen 14 of the apparatus 10 has a see-through region for allowing theuser to view the patient directly, the video may be presented directlyover the patient as seen by the user. This way, the user can visualizehow the movement of an object will impact the patient. Also, in somecases, the virtual dry-run may also include a simulation of energydelivery. For example, simulated doses from different energy deliverydirections may be tracked and accumulated at target region to determinea simulated dose effect on the target region and/or at critical organnext to the target region.

FIG. 2A illustrates a treatment system being used with the apparatus ofFIG. 1. The radiation system 200 includes a structure 212 having a firstside 214, and a second side 216. In the illustrated embodiments, thestructure 212 has a through bore sized for accommodating at least a partof a patient. The through bore of the structure 212 provides a passagefor allowing at least a portion of a patient to be transported from oneside of the structure 212 to an opposite side of the structure 212. Insome embodiments, a diagnostic procedure (e.g., an imaging procedure) isperformed on the patient on one side of the structure 212 (e.g., for thepurpose of obtaining information, such as a position of a target region,of the patient), and the patient is then transported through the bore tothe opposite side of the structure 212 for a treatment procedure. Inother embodiments, the patient is treated on one side of the structure212, and is then transported through the bore to the opposite side ofthe structure 212 for further procedure(s), such as a diagnosticprocedure (e.g., to evaluate a treatment procedure, or to verifylocation, orientation, and/or shape of a target tissue,) or a treatmentprocedure.

It should be noted that the shape and configuration of the structure 212should not be limited to the example discussed previously, and that thestructure 212 can have other configurations in other embodiments. Forexample, in other embodiments, the structure 212 can have a curvilinearshape, or other shapes. Also, in some embodiments, the structure 212 canhave a size and shape such that the structure can house mechanical andelectrical components associated with an operation of the radiationsystem 200 as desired. The radiation system 200 also includes aradiation source 240 (located closer to the first side 214 than thesecond side 216) for delivering a radiation beam. The radiation beam canbe a pencil beam, a fan beam, a cone beam, or other types of beamshaving different configurations. As used in this specification, the term“radiation source” refers to an emission point/region of a radiationbeam (e.g., radiation beam), and may or may not include components, suchas a particle generator, an accelerator, a cooling system, a shielding,etc., that are used to generate the radiation beam. As shown in thefigure, the radiation system 200 includes an arm 230 secured to thestructure 212, and the first radiation source 240 is secured to the arm230. Some or all of the components used to generate the radiation beamcan be housed within the arm 230, the structure 212, a separate housing(not shown), or combination thereof. For example, in some embodiments,the accelerator associated with the radiation source 240 may be housedwithin the arm 230. In such cases, one or more magnets (electromagnet(s)or permanent magnet(s)) may be provided within the arm 230 for changinga characteristic (e.g., a trajectory) of an electron beam created by theaccelerator. If permanent magnet(s) is used, its associated magneticfield can be trimmed electromagnetically (e.g., using one or moreelectromagnetic coil(s)) or mechanically (e.g., using one or morepermanent magnet(s)). Also, in some embodiments, the mechanical trimmingcan be performed using a magnetic shunt.

In other embodiments, or any of the embodiments described herein, theradiation system 200 may not include the arm 230. In such cases, thefirst radiation source 240 may be rotatably secured to the structure212. For example, the radiation source 240 may be secured to a ring(which may be a full ring or a partial ring) that is rotatable relativeto the structure 212 in a slip-ring configuration. In such cases, atleast some of the components within arm 230 may be disposed within thestructure 212.

In the illustrated embodiments, the radiation source 240 is a treatmentradiation source for providing treatment energy. In such cases, theradiation system 200 further includes one or more collimators (notshown) for controlling a delivery of the radiation beam (e.g., changinga shape of the beam). A collimator can be, for example, a multi-leafcollimator, which is known in the art. Alternatively, the radiationsource 240 can be a diagnostic radiation source for providing diagnosticenergy. In some embodiments, the treatment energy is generally thoseenergies of 160 keV or greater, and more typically 1 MeV or greater, anddiagnostic energy is generally those energies below the high energyrange, and more typically below 160 keV. In other embodiments, thetreatment energy and the diagnostic energy can have other energy levels,and refer to energies that are used for treatment and diagnosticpurposes, respectively. For example, a radiation beam having an energylevel that is typically used for treatment purpose may be considered ashaving a diagnostic energy level if the radiation beam is used fordiagnostic purpose (e.g., for imaging). As such, the term “treatmentenergy” and the term “diagnostic energy” should not be limited to energylevels having certain magnitudes. In further embodiments, the radiationsource 240 is a multi-energy x-ray source that is capable of providingradiation energy at different energy levels. By way of example, theradiation source 240 is able to generate X-ray radiation at a pluralityof photon energy levels within a range anywhere between approximately 10kilo-electron-volts (keV) and approximately 20 mega-electron-volts(MeV). Radiation sources capable of generating X-ray radiation atdifferent energy levels are described in U.S. Pat. No. 6,888,919,entitled “RADIOTHERAPY APPARATUS EQUIPPED WITH AN ARTICULABLE GANTRY FORPOSITIONING AN IMAGING UNIT,” issued on May 3, 2005, and U.S. Pat. No.7,649,981, entitled “MULTI-ENERGY X-RAY SOURCE,” issued on Jan. 19,2010, both of which are expressly incorporated by reference in theirentirety.

In the illustrated embodiments, the radiation system 200 furtherincludes a control system 278. The control system 278 includes aprocessor 284, such as a computer processor, coupled to a control 280.The control system 278 may also include a monitor 286 for displayingdata and an input device 288, such as a keyboard or a mouse, forinputting data. In some embodiments, during an operation of theradiation system 200, the radiation source 240 rotates about the patient(e.g., as in an arc-therapy). The rotation and the operation of theradiation source 240 are controlled by the control 280, which providespower and timing signals to the radiation source 240 and controls arotational speed and position of the radiation source 240 based onsignals received from the processor 284. Although the control 280 isshown as a separate component from the structure 212 and the processor284, in alternative embodiments, the control 280 can be a part of thestructure 212 or the processor 284.

In any of the embodiments described herein, the radiation system 200 canfurther include an imager located next to the first opening 218 andopposite from the radiation source 240. Such imager may be configured tofunction as an on-board imager during a treatment procedure.

It should be noted that the radiation system 200 should not be limitedto the configuration discussed previously, and that the radiation system200 can have other configurations in other embodiments.

As shown in FIG. 2A, the radiation system 200 further includes a patientsupport system 290 for supporting and positioning the patient. Thepatient support system 290 includes a base 300 rotatably coupled to afloor so that the base 300 is ratatable about a vertical axis 350. Thepatient support system 290 also includes a support structure 302rotatably coupled to the base 300 so that the support structure 302 isrotatable relative to the base 300 about another vertical axis 352. Thepatient support system 290 also includes an arm system 208 having afirst arm 366 and a second arm 368. The arm system 208 is coupled to thesupport structure 302 and a patient support 201. The arms 366, 368 areconfigured to move in correspondence with respect to each other tochange an elevation of the patient support 201. The patient supportsystem 290 further includes a positioner 310 configured to translate thepatient support 201 along a longitudinal axis of the patient support201.

During a treatment procedure, the patient support system 290 canposition the patient in various degrees of freedom. For example, thepatient support system 290 can be operated to rotate the patient aboutthe axis 350 and/or the axis 352 to place the patient at differentco-planar positions with respect to the treatment machine. The patientsupport system 290 can also varies the height or elevation of thepatient in synchronization with a position of the radiation source 240.Also, the patient support system 290 can translate the patienthorizontally to move the patient to an operative position associatedwith the radiation source 240. The patient support system 290 can alsotranslate the patient horizontally to move the patient into an opening222 at the arm 230, and through a bore at the structure 212 to reach anoperative position associated with another medical device 266 on thesecond side 216 of the structure 212. The medical device 266 may be animaging device (such as a CT device, a MRI device, a x-ray device,etc.), or a treatment device.

In some embodiments, the radiation system 200 can further include ax-ray source 251 and an imager 252 secured to the arm 230 (FIG. 2B),wherein the x-ray tube 251 and the imager 252 are positioned to image atleast a portion of the patient. The x-ray tube 251 and the imager 252can be used to generate data regarding a patient while the patient ispositioned in an operative position associated with the radiation source240. For example, in some embodiments, the x-ray tube 251 generates acone beam, and the imager 252 generates cone beam CT data, whichrepresent image of a portion of a patient. Alternatively, the imagingdevices can be used for radiography or fluoroscopic imaging. In someembodiments in which the radiation system 200 does not include the arm230, but includes a ring gantry instead, the x-ray tube 251 and theimager 252 could be attached to the ring gantry.

As shown in FIG. 2B, the radiation system 200 can further include animager 253 located opposite from the radiation source 240. Such imager253 may be configured to function as an on-board imager during atreatment procedure.

FIGS. 3A-3B illustrate an example of the apparatus 10 providing agraphical representation of medical information in an overlayconfiguration with respect to a patient or an image (e.g., a real-timeimage) of the patient, while the patient is positioned next to atreatment device. In the illustrated example, the treatment device isthe radiation system 200 of FIG. 2B. However, in other embodiments, thetreatment device may be any of other medical treatment devices. As shownin FIG. 3A, the user 400 is wearing the apparatus 10. The user 400 cansee the patient 410 while the patient 410 is being supported on thepatient support 201 next to the radiation system 200. The user 410 canalso see other objects surrounding the patient 410 via the apparatus 10.

In some embodiments, the screen 14 is transparent, and so the user cansee the patient 410 directly through the transparent screen 14. In otherembodiments, the screen 14 may be a digital display that is a part of avirtual-reality device. In such cases, the user cannot view through thescreen 14 to see the real-world. Instead, the graphics generator 30 mayprovide images of the patient 410 continuously in real-time. In somecases, the images of the patient 410 may be generated based on signalstransmitted from an optical device (e.g., a camera).

Also, as shown in FIG. 3A and FIG. 3B, the user 400 can see medicalinformation 420 as provided by the screen 14 of the apparatus 10. In theillustrated example, the medical information 420 is dose (e.g.,delivered dose, predicted dose, and/or planned dose). In such cases, thegraphics generator 30 provides a graphical representation of the dosefor display on the screen 14, so that when the user view through thescreen 14 to see the patient 410, the dose graphics appears in anoverlay configuration with respect to the patient 410. As the user 400moves his/her head to change the viewing direction, the graphicalrepresentation of the dose as appeared on the screen 14 will also changecorrespondingly (e.g., in response to the variable viewing direction ofthe user 400). For example, as the user 400 changes the viewingdirection to view another part of the patient 410, the graphicsgenerator 30 will correspondingly change the medical information so thatthe user can see the dose information for the other part of the patient410. In other cases, the user 400 can view the same part of the patient,but from a different viewing direction. In such cases, the graphicalrepresentation of the dose as appeared on the screen 14 will also changecorrespondingly.

In some embodiments, the dose image as rendered and displayed on thescreen 14 of the apparatus 10 may be configurable based on user'spreference or selection. For example, a user may use a user interface(e.g., which may be implemented at the apparatus 10, such as one or morebuttons at the goggle) to select a direction of rendering for the doseimage. In some cases, the user may instruct the processing unit 12 ofthe apparatus 10 to render the dose image in a direction that is alongone or more isocenter axes. In other cases, the user may instruct theprocessing unit 12 of the apparatus 10 to render the dose image in adirection that is perpendicular to a viewing direction of the user.

As can be seen from the above example, the apparatus 10 is advantageousbecause it allows the user 400 to see medical information 420 in anoverlay configuration with respect to the patient in real-time. This canoccur when the user 400 is setting up the patient, reviewing delivereddose after a treatment delivery, setting up the treatment machine for anext treatment delivery, reviewing a treatment plan, and/or adjustingthe treatment plan. Without the apparatus 10, the user 400 can only seethe patient 410, and there is no medical information available for theuser 400 to view while the user is looking at the patient 410 (FIG. 3C).

In the example shown in FIG. 3A, there is only one user 400 wearing theapparatus 10. In other embodiments, there may be multiple users 400wearing corresponding apparatuses 10.

In the above example, the dose information may be considered to be anexample of medical information. In other example, the medicalinformation may be image data of the patient. By means of non-limitingexamples, the image data may be CT image, digital x-ray image,ultrasound image, MRI image, PET image, PET-CT image, SPECT image,SPECT-CT image, etc.

FIGS. 4A-4B illustrates another example of the apparatus 10 providing agraphical representation of medical information in an overlayconfiguration with respect to a patient or an image (e.g., a real-timeimage) of the patient, while the patient is positioned next to atreatment device. In the illustrated example, the treatment device isthe radiation system 200 of FIG. 2B. However, in other embodiments, thetreatment device may be any of other medical treatment devices. As shownin FIG. 4A, the user 400 is wearing the apparatus 10. The user 400 cansee the patient 410 while the patient 410 is being supported on thepatient support 201 next to the radiation system 200. The user 410 canalso see other objects surrounding the patient 410 via the apparatus 10.

Also, as shown in FIG. 4A and FIG. 4B, the user 400 can see medicalinformation 420 as provided by the screen 14 of the apparatus 10. In theillustrated example, the medical information 420 is internal image (CTimage) of the patient 410. In such cases, the graphics generator 30provides the internal image for display on the screen 14, so that whenthe user view through the screen 14 to see the patient 410, the internalimage appears in an overlay configuration with respect to the patient410. As the user 400 moves his/her head to change the viewing direction,the internal image as appeared on the screen 14 will also changecorrespondingly (e.g., in response to the variable viewing direction ofthe user 400). For example, as the user 400 changes the viewingdirection to view another part of the patient 410, the graphicsgenerator 30 will correspondingly change the medical information so thatthe user can see the internal image for the other part of the patient410. In other cases, the user 400 can view the same part of the patient,but from a different viewing direction. In such cases, the internalimage of the patient 410 as appeared on the screen 14 will also changecorrespondingly.

In some embodiments, the CT image as rendered and displayed on thescreen 14 of the apparatus 10 may be configurable based on user'spreference or selection. For example, a user may user a user interface(e.g., which may be implemented at the apparatus 10, such as one or morebuttons at the goggle) to select a direction of rendering for the CTimage. In some cases, the user may instruct the processing unit 12 ofthe apparatus 10 to render the CT image in a direction that is along oneor more isocenter axes. In other cases, the user may instruct theprocessing unit 12 of the apparatus 10 to render the CT image in adirection that is perpendicular to a viewing direction of the user.Also, the user may instruct the processing unit 12 to provide surfacerendering, which shows organ surfaces. In other cases, the user mayinstruct the processing unit 12 to provide cross sectional view of theinternal organs of the patient 410.

In the above example, the medical information is image data thatcomprises CT image. In other embodiments, the image data may be digitalx-ray image, ultrasound image, MRI image, PET image, PET-CT image, SPECTimage, SPECT-CT image, etc.

As can be seen from the above example, the apparatus 10 is advantageousbecause it allows the user 400 to see medical information 420 in anoverlay configuration with respect to the patient in real-time. This canoccur when the user 400 is setting up the patient, reviewing delivereddose after a treatment delivery, setting up the treatment machine for anext treatment delivery, reviewing a treatment plan, and/or adjustingthe treatment plan. Without the apparatus 10, the user 400 can only seethe patient 410, and there is no medical information available for theuser 400 to view while the user is looking at the patient 410 (FIG. 4C).

In the example shown in FIG. 4A, there is only one user 400 wearing theapparatus 10. In other embodiments, there may be multiple users 400wearing corresponding apparatuses 10.

In one or more embodiments described herein, the processing unit 12 isconfigured to align the graphics as displayed on the screen 14 with acertain part of the patient, or with a certain part of an image of thepatient. This way, as the user of the apparatus 10 changes his/herviewing direction, the graphics will change in real-time and will remainaligned with the correct part of the patient or the correct part of theimage of the patient. In one implementation, the apparatus 10 may beconfigured to detect certain part(s) of the patient in real-time. Suchmay be accomplished using one or more cameras to view the patient.Images from the camera(s) may then be processed by the processing unit12 to determine the position(s) of certain part(s) of the patient. Insome cases, markers may be placed at the patient to facilitate theaccomplishment of such purpose. In other cases, anatomical landmarks atthe patient may be utilized as markers. In other embodiments, thecamera(s) may be depth camera(s) for detecting the surface of thepatient. The detected surface may then be utilized by the processingunit 12 to identify the position of the patient (e.g., position(s) ofcertain part(s) of the patient). Once the actual position of the certainpart(s) of the patient has been determined, the processing unit 12 thendetermines a position of the graphics (representing certain medicalinformation) with respect to the determined actual position. Theposition of the graphics may then be utilized by the processing unit 12for correct positioning of the graphics at the right location of thescreen 14. For example, if the medical information comprises an image ofan internal part of the patient, the position of the internal part ofthe patient with respect to certain part P of the patient is known, ormay be derived from analysis of the image. During use of the apparatus10, the processing unit 12 analyzes real-time images of the patient todetermine the actual position of the same part P of the patient. Basedon the known relative positioning between the image of the internal partof the patient and the certain part P of the patient, then processingunit 12 then places the graphics (representing the same internal part ofthe patient) at the same relative position with respect to the actualposition of the certain part P of the patient at the screen 14 inreal-time.

FIG. 5 illustrates information flow for the apparatus of FIG. 1. Asshown in the figure, the processing unit 12 of the apparatus 10 mayreceive various information, and may be configured to process suchinformation for display at the screen 14 of the apparatus 10. Inparticular, the processing unit 12 may have one or more input(s) forreceiving patient information 510, such as diagnostic image 512 orpatient's physical scan 514. The image or the scan may be a CT image, aPET image, a x-ray, a MRI, a PET-CT, an ultrasound image, etc. Also,patient information 510 may comprise any information regarding a featureof a patient. In some cases, one or more camera(s) may be employed toobtain images of the patient, and the images are processed to obtainpatient information.

The processing unit 12 may also have one or more input(s) for receivingdose information 520 (such as estimated dose, actual dose, planned dose,etc.). In one implementation, the dose information 520 may be obtainedfrom a dose calculation engine 522. Also, patient information 510, suchas images of the patient, patient's weight, patient's height, dimension,etc., may also be received from the dose calculation engine 522 in someembodiments.

The processing unit 12 may further include one or more input(s) forreceiving room information 540. By means of non-limiting examples, theroom information 540 may include information regarding a position of anobject in a room, a physical dimension and shape of an object in a room,a model representing an object in a room, an image of an object in aroom, a feature associated with an object in a room, or any combinationof the foregoing, wherein the object may be a wall, a ceiling, a floor,a device in the room, etc. In some cases, the room information 540 maybe isocenter location associated with the treatment machine. Also, insome embodiments, room information may comprise information regardingany feature in a room. In one implementation, one or more camera(s) maybe employed to obtain images of object(s) in a room, and the images areprocessed to obtain the room information.

In some embodiments, the same input at the processing unit 12 may beutilized to receive multiple information, such as patient information510 and dose information 520, or patient information 510 and roominformation 540, or room information 540 and dose information 520, orpatient information 510, room information 540, and dose information 520.

It should be noted that the apparatus 10 is not limited to a wearabledevice that is in a form of goggle or glasses. In other embodiments, theapparatus 10 may be in a form of a helmet, hood, facemask, etc., that isfor worn at the head of the user. In other embodiments, the apparatus 10may be in a form of a handheld device. By means of non-limitingexamples, the handheld device may be a cell phone, an iPad, a miniPad, atablet, etc.

As discussed, in some embodiments, the processing unit 12 of theapparatus 10 may obtain patient information and/or room information froma camera. The camera may be fixedly secured to the apparatus 10. Inother embodiments, the camera may be secured to a patient support. FIG.6 illustrates a treatment system 600 having a camera system that may beused with the apparatus 10 of FIG. 1. As shown in the figure, thetreatment system 600 includes a treatment machine 602 and a patientsupport 604. A camera 610 a is secured to the patient support 604. Thetreatment machine 602 may be the same as that shown in FIG. 2B in someembodiments. During use, the camera 610 a generates images of thepatient supported on the patient support 604 and images of objectssurrounding the patient in real time. The images are transmitted to theprocessing unit 12 of the apparatus 10 of FIG. 1, which processes theimages to obtain patient information and/or room information. As shownin FIG. 6, in some embodiments, the treatment system 600 may alsoinclude additional cameras 610 b, 610 c for obtaining patientinformation (e.g., images of the patient) and/or room information (e.g.,images of components of the treatment machine 602 in a treatment room).

In other embodiments, there may be multiple cameras. For example, theapparatus 10 may have multiple cameras that are fixedly secured to theapparatus 10. For example, if the apparatus 10 is in the form of goggle,then the cameras may be secured to a frame of the goggle. In otherembodiments, there may be multiple cameras attached to different objectsin a treatment room. In such cases, the cameras provide images of theobjects in the treatment room in real time, and transmit the images tothe processing unit 12 of the apparatus 10. In further embodiments,there may be one or more camera(s) attached to the apparatus 10, and oneor more camera(s) attached to different objects in the treatment room.During use, the camera(s) at the apparatus 10 and the other camera(s) inthe treatment room provide real time images to the processing unit 12,which processes the images to determine patient information and/or roominformation.

FIG. 7 illustrates a method 700 in accordance with some embodiments. Themethod 700 may be performed by an apparatus, such as the apparatus 10,in a medical process. The medical process may be a treatment processand/or an imaging process. Also, the treatment process may be aradiation treatment process. In other embodiments, the treatment processmay involve other types of energy that is different from radiation. Bymeans of non-limiting examples, the treatment energy involved in thetreatment process may be radiofrequency energy, thermal energy,ultrasound energy, proton energy, electron energy, etc. Also, in someembodiments, the medical process may involve a particle accelerator. Forexample, the medical process may be a proton treatment process, aradiation therapy, a x-ray process, a CT imaging process, etc., each ofwhich involves use of a particle accelerator. Referring to the figure,the method 700 includes: obtaining, by a processing unit of theapparatus, medical information (item 702); obtaining, by the processingunit of the apparatus, a viewing direction of a user of the apparatus(item 704); processing, by the processing unit of the apparatus, themedical information based on the viewing direction of the user of theapparatus to create a graphical representation of the medicalinformation for presentation to the user (item 706); and displaying thegraphical representation in a screen of the apparatus (item 708). Insome embodiments, the processing unit involved in items 702, 704, 706may be the processing unit 12 of the apparatus 10. In other embodiments,the processing unit involved in items 702, 704, 706 may be any otherprocessing unit.

In some embodiments, the act of obtaining the medical information initem 702 may be performed by the processing unit receiving the medicalinformation from another component. In other embodiments, the act ofobtaining the medical information in item 702 may be performed by theprocessing unit determining (e.g., deriving) the medical informationbased on data received by the processing unit.

Also, in some embodiments, the act of obtaining the viewing direction ofthe user of the apparatus in item 704 may be performed by the processingunit receiving the viewing direction from another component, such as anorientation sensor. The orientation sensor may be coupled to the samedevice that contains the processing unit, or may be separate from theprocessing unit. In other embodiments, the act of obtaining the viewingdirection in item 704 may be performed by the processing unitdetermining the viewing direction based on data received by theprocessing unit.

In addition, in some embodiments, the act of processing the medicalinformation based on the viewing direction of the user of the apparatusto create the graphical representation of the medical information initem 706 may be performed by the a graphics generator of a processingunit (e.g., the processing unit 12), which determines a configuration ofthe graphical representation based on the viewing direction of the user.In some embodiments, the medical information comprises image data, suchas CT data. In such cases, the processing unit creates a CT imagerendering based on the viewing direction, so that when the CT imagerendering is displayed in a screen, the CT image rendering willcorrespond with a position of a patient as directly viewed by the user,or will correspond with a position of an image of the patient as viewedby the user. In some embodiments, the processing unit is configured tocreate the cross section of the CT image along isocenter axes. In otherembodiments, the processing unit is configured to create the crosssection of the CT image along a direction that is orthogonal to theviewing direction of the user.

In other embodiments, the medical information may comprise a depth of atreatment isocenter. In such cases, the depth of the treatment isocentermay be rendered by the processing unit over a patient (e.g., withrespect to a viewing direction of the user of the apparatus 10), or fordisplay in an overlay configuration with an image (e.g., a real-timeimage) of the patient.

In further embodiments, the medical information may be any of otherinformation described herein. By means of non-limiting examples, themedical information may be information or data regarding a feature of apatient, an actual position of the patient, a desired position of thepatient, a model of a patient, etc.

In some embodiments, the act of displaying the graphical representationin item 708 may be performed by a screen that is partially or completelytransparent. Such configuration allows a user of the apparatus todirectly view the real world. In other embodiments, the act ofdisplaying the graphical representation in item 708 may be performed bya screen that is a digital display of a virtual-reality device. In oneimplementation, the act of displaying the graphical representation maybe performed by the screen 14 of the apparatus 10.

Guidance on Treatment Plan Adaptation

In one or more embodiments described herein, the apparatus 10 may befurther configured to provide guidance on treatment plan adaptation. Forexample, the processing unit 12 may be configured to provide guidancefor display on the screen 14, wherein the guidance may assist a user ofthe apparatus 10 in deciding whether or not to execute radiationtreatment for the patient. In one implementation, guidance informationmay be received by the processing unit 12 (e.g., wirelessly or via awire or cable). The processing unit 12 of the apparatus 10 may thenprocess the guidance information for display on the screen 14. Theguidance information may be determined online or offline.

In some embodiments, a physician in a different room from the treatmentroom may view a real-time image of the patient while the patient is onthe patient support, and evaluate the treatment configuration todetermine whether to proceed with the treatment or not. If the physiciandetermines that treatment may be proceeded, the physician may thengenerate a signal via an electronic user interface. The signal is thentransmitted to the apparatus 10 for display on the screen 14. Forexample, the signal may be “Proceed with treatment”. Alternatively, ifthe physician determines that the treatment is not to be proceeded, thephysician may then user the electronic user interface to generate asignal (e.g., a signal indicating “Not to proceed with treatment”),which is then transmitted to the apparatus 10 for display on the screen14.

In further embodiments, instead of having the physician who is in adifferent room from the treatment room providing input for planadaptation, the user wearing the apparatus 10 may himself/herselfdetermines a treatment plan, and/or determine whether to adapt atreatment plan, based on guidance information displayed on the screen14. For example, in some cases, a current treatment plan may be loadedinto the treatment device while the patient is supported on the patientsupport. The user of the apparatus 10 may then determine a new treatmentplan (different from the current treatment plan) by selecting apre-determined treatment plan from a plurality of pre-determinedtreatment plans. In one implementation, the selection may be performedbased on a guidance information presented on the screen 14 of theapparatus 10. For example, there may be a first treatment plan for thepatient that was generated for non-filled bladder, and a secondtreatment plan for the patient that was generated for filled bladder. Insuch example, the processing unit 12 of the apparatus 10 may output amessage indicating that the patient has a filled bladder for display onthe screen 14. When the user of the apparatus 10 sees the message, theuser may then select the treatment plan that is designed for the patientwith the filled bladder. In some embodiments, the processing unit 12 mayprovide the guidance information (e.g., the message indicating that thepatient has a filled bladder) in response to processing of a patientnote or nurse's or doctor's notes, which indicates that the patient hasa filled bladder. Alternatively, the processing unit 12 may beconfigured to obtain a real-time image of the patient (e.g., CT image,x-ray, ultrasound image, MRI image, etc.) and analyze the image todetermine whether the patient has a filled bladder or not.

In other embodiments, instead of selecting one of the pre-determinedtreatment plans as the new treatment plan, the user of the apparatus 10may determine the new treatment plan by changing a parameter in thecurrent treatment plan to obtain a different treatment plan.

It should be noted that the guidance information provided by theprocessing unit 12 is not necessarily based on bladder condition of thepatient, and that the processing unit 12 may be configured to provideguidance information based on other parameter(s). For example, in otherembodiments, dose variations may be overlayed on the patient's tumor andcritical organs when displayed on the screen 14, so that the treater canadjust the patient's treatment plan accordingly. As another example, theguidance information may be any information that allows the user toidentify a change to the patient's anatomy. In one implementation, theprocessing unit 12 may be configured to display one or more planningstructures on the screen 14 so that the user of the apparatus 10 can seethe planning structures in an overlay configuration with the currentconfiguration(s) of the patient, wherein the current configuration(s)may be the actual shape and geometry of the patient as viewed throughthe screen 14, a constructed image of an external part of the patient,or an image of an internal part of the patient. This way, if there is achange to the patient's anatomy (e.g., due to weight loss), then usermay then notice the change through viewing of the screen 14.

As a further example, the processing unit 12 may be configured toprocess one or more current images (e.g., real-time image(s), orimage(s) obtained while the patient is on the patient support next tothe treatment machine) of the patient by comparing the current image(s)with planning image(s). The processing unit 12 may provide a result ofthe image comparison for display on the screen 14, wherein the resultmay be an example of guidance information that provides guidance fortreatment plan determination and/or adaptation. The image comparison maybe achieved by the processing unit 12 performing image shape analysis(e.g., shape recognition and shape comparison, etc.), image analysis(e.g., image correlation), or both. If the current image(s) differ bythe planning image(s) and the difference exceeds a certain threshold,the processing unit 12 may then generate a message to inform the user ofthe apparatus of such difference for display on the screen 14. By meansof non-limiting examples, the processing unit 12 may be configured todetect difference in size of target, shape of target, position oftarget, size of critical organ, shape of critical organ, position ofcritical organ, or any combination of the foregoing, based on acomparison between current image(s) and planning image(s). The currentimage(s) may be CT image(s), x-ray image(s), MRI image(s), ultrasoundimage(s), etc. Similarly, the planning image(s) may be CT image(s),x-ray image(s), MRI image(s), ultrasound image(s), etc. In someembodiments, the current image(s) may be of the same type as theplanning image(s). In other embodiments, the current image(s) may be ofdifferent type as that of the planning image(s).

In other embodiments, instead of having the processing unit 12 performimage comparison between the current image and the planning image, theprocessing unit 12 may simply processes the current image with theplanning image for simultaneous display on the screen 14. The display ofthe planning image and/or the current image may be considered as anexample of guidance information, which provides guidance for treatmentplan determination and/or adaptation. The current image may be displayedwith the planning image in an overlay configuration. This way, the userof the apparatus 10 may notice any difference between the current imageand the planning image. In one implementation, the processing unit 12may identify an area in the current image that is larger than theplanning image, and may provide a first color for such area. Theprocessing unit 12 may also identify an area in the planning image thatis larger than the current image, and may provide a second color forsuch area. This way, when the colored areas are displayed on the screen14, the user of the apparatus 10 will see any changes in the size andshape of the internal organ of the patient. In some cases, both of thecurrent image and the planning image may be displayed over the patientas viewed through the screen 14, or over an image of the patient who isbeing supported on the patient support, based on a viewing orientationand position of the apparatus 10.

In further embodiments, the processing unit 12 may also be configured toidentify a change in a patient's posture, and/or to provide informationfor allowing the user of the apparatus 10 to detect a change in thepatient's posture. For example, in some embodiments, the processing unit12 may be configured to generate an image of the patient, showing theposture of the patient for the treatment plan, for display on the screen14 of the apparatus 10. The planning posture of the patient may bedisplayed directly over the patient as viewed through the screen 14, orin an overlay configuration with a real-time image of the patient asdisplayed on the screen 14. This allows the user of the apparatus 10 tosee any discrepancy between the planning posture of the patient and thecurrent posture of the patient while the patient is being supported onthe patient support next to the treatment machine. In response to thedetected discrepancy, the user of the apparatus 10 may position thepatient or may instruct the patient to change posture, so that thecurrent posture of the patient will match the planning posture of thepatient. In one implementation, the planning posture of the patient maybe represented by a constructed image that is output by the processingunit 12 based on a viewing direction and position of the apparatus 10,wherein the constructed image may be considered as an example ofguidance information, which provides guidance for treatment plandetermination and/or adaptation. For example, if the user of theapparatus 14 is standing towards the right side of the patient, then theprocessing unit 12 will generate the image indicating the planningposture as viewed from the right side of the patient. This way, when theconstructed image is displayed on the screen 14 in an overlayconfiguration with the patient, the planning posture can be comparedwith the current posture from the same viewing direction and position.

In other embodiments, the processing unit 12 may be configured to detectthe discrepancy between planning posture and current posture of thepatient. For example, in some cases, the processing unit 12 may comparea planning image of the patient with the current image of the patient todetermine if a difference in the posture exceeds a certain threshold. Ifthe difference exceeds the threshold, the processing unit 12 may thengenerate a message for display on the screen 14 to inform the user ofthe apparatus 10. For example, if the patient's hand is incorrectlyplaced (which may impact treatment beam delivery, and may lead to anincorrect dose delivery), then the message will inform the user of theapparatus 10 of such. In some cases, the message may also indicate thedetails of the posture difference, such as, which part of the patient isincorrectly positioned, how much (the distance) the patient part isincorrectly positioned, etc. The message provided by the processing unit12 may be considered as an example of guidance information, which guidesthe user in determining a treatment plan and/or adaptation of atreatment plan. The current image of the patient may be an opticalcamera image, a depth image, or any other types of image that indicatesa feature of an exterior part of the patient. Alternatively, oradditionally, the current image may indicate an interior part of thepatient. Similarly, the planning image of the patient may be an opticalcamera image, a depth image, a constructed part of a three-dimensionalimage, or any other types of image that indicates a feature of anexterior part of the patient. Alternatively, or additionally, thecurrent image may indicate an interior part of the patient.

In further embodiments, the processing unit 12 may also generate one ormore images indicating treatment beam angles and positions to beachieved, and provide such image(s) for display on the screen 14 of theapparatus 10. In the illustrated embodiments, the images indicatingtreatment beam angles and positions may be based on viewing orientationand position of the apparatus 10. In one implementation, the images maybe one or more lines representing one or more beam trajectories for oneor more respective beams to be delivered from respective gantryangle(s). Optionally, the processing unit 12 may also provide an imageof a target to receive the radiation beam(s) for display on the screen14. In some embodiments, the lines representing the beam trajectoriesmay be displayed over the patient as seen through the screen 14, or maybe displayed in an overlay configuration with a real-time image of thepatient as displayed on the screen 14. This allows the user of theapparatus 10 to see if any of the lines representing the radiation beamsundesirably intersects a part of the patient. For example, if a linerepresenting the radiation beam to be delivered undesirably intersects apatient's arm, the user of the apparatus 10 may then determines that thetreatment configuration is not desirable. The user may in such caseselect a new treatment plan, adjust a parameter of the current treatmentplan to obtain a new treatment plan, or may change the posture of thepatient. In the above embodiments, the images representing the beamangles and positions may be considered as an example of guidanceinformation that guides the user to determine a treatment plan and/or toadapt a treatment plan.

It should be noted that guidance information provided by the processingunit 12 for display on the screen 14 may include any treatment planinformation. This way, when the user of the apparatus 10 sees thetreatment plan information on the screen 14, the user may utilize suchinformation to determine how to adjust the patient to adapt thetreatment plan, and/or may utilize such information to determine whetherthe treatment plan can be executed. If it is determined that thetreatment plan cannot be executed, then the user may determine a newtreatment plan accordingly.

The treatment plan information may be any information that is involvedin the treatment of the patient. The treatment plan information may beany parameter of a treatment plan, or may be any information derivedfrom a treatment plan. For example, in some embodiments, the treatmentplan information may comprise a position of an energy source fordelivering a treatment beam. In such cases, the graphical representationprovided by the processing unit 12 may comprise a line representing atrajectory of the treatment beam. In another example, the treatment planinformation comprises an expected configuration of a component of atreatment machine. In some cases, the expected configuration maycomprise an expected position of the component (e.g., a gantry, anenergy source, etc.) of the treatment machine. In another example, thetreatment plan information may comprise an expected dose for an internaltarget of a patient. In such cases, the graphical representationprovided by the processing unit 12 may indicate the expected dosegraphically, and the graphical representation may be displayed over apatient as viewed through the display, or may be displayed in an overlayconfiguration with an image of the patient, so that the graphicalrepresentation is at a location that corresponds with a position of theinternal target of the patient. In a further example, the treatment planinformation comprises an expected posture of a patient. The treatmentplan information may also comprise a target position, a target size, atarget shape, a critical organ position, a critical organ size, acritical organ shape, or any combination of the foregoing. In anotherexample, the treatment plan information may comprise a target fluence,and the processing unit 12 may be configured to provide the graphicalrepresentation for representing the target fluence. In another example,the treatment plan information may comprise a trajectory of a componentof a treatment machine. In such cases, the graphical representationprovided by the processing unit 12 may be configured to indicate thetrajectory of the component of the treatment machine.

In some embodiments, the processing unit 12 may be configured tosimulate a treatment based on the treatment plan information. In suchcases, the graphical representation provided by the processing unit 12for display on the screen 14 may comprise one or more images representedthe simulated treatment. In some cases, the one or more images may be asequence of images forming a video. Also, in some embodiments, each ofthe images in the video may be based on a viewing direction and positionof the user of the apparatus 10. The simulated treatment may comprise asimulated movement of a component of a treatment machine, such as thegantry, the energy source, the patient support, etc. In some cases, thegraphical representation provided by the processing unit 12 for displayon the screen 14 may comprise a video showing the simulated movement ofthe component of the treatment machine.

Also, in some embodiments, the simulated movements may be based on theactual condition (e.g., position, shape, etc.) of the objects asdetected in the treatment room. For example, the simulated movement ofthe object may be conducted so that the movement path begins at, orintersects with, the actual position of the object. The processing unit12 may also provide graphics indicating the simulated movements of theobjects for display on the screen 14 of the apparatus 10. For example,the processing unit 12 may provide a video showing how an object movesrelative to the patient. In the embodiment in which the screen 14 of theapparatus 10 has a see-through region for allowing the user to view thepatient directly, the video may be presented directly over the patientas seen by the user. This way, the user can visualize how the movementof an object will impact the patient. Also, in some cases, theprocessing unit 12 may perform a simulation of energy delivery. Forexample, simulated doses from different energy delivery directions maybe tracked and accumulated at target region to determine a simulateddose effect on the target region and/or at critical organ next to thetarget region.

In further embodiments, the processing unit 12 may be configured toprovide virtual light field for display on the screen 14 of theapparatus 10. For example, in some cases, virtual light field may bedisplayed to allow a user of the apparatus 10 to visualize surfacefluence for a simulated treatment or for an actual treatment. Thevirtual light field may also allow the user to see if radiation is goingwhere it should. As another examples, the virtual light may representfluence painting, or may represent skin flash.

In some embodiments, the processing unit 12 may also be configured toprovide information regarding fluence virtualization for display on thescreen 14.

In some embodiments, the apparatus 10 further includes a user interfacefor allowing the user to determine a new treatment plan by selecting thenew treatment plan from a plurality of pre-determined treatment plans,while a patient is supported on a patient support, or by changing aparameter of a current treatment plan, while a patient is supported on apatient support. The user interface may include an input device, such asone or more buttons, one or more switches, one or more dials, a touchscreen, a trackball, a joystick, etc. In some cases, the input devicemay be located at a frame of a goggle, a pair of glasses, visor,headgear, etc. In other cases, the input device may be implemented on aseparate unit that is communicatively coupled to the wearable structureof the apparatus 10. The separate unit may be a handheld device, or maybe any device that can be placed in a pocket or be clipped to a belt orclothing. In one implementation, the separate unit may be implementedusing a cell phone, such as an iPhone, a smart phone, etc.

In some embodiments, the user interface may also be configured forallowing the user to control a position of an energy source, a patientsupport, one or more camera(s), one or more alignment laser(s), one ormore light(s), a calibration of a device, a speaker for communicationwith a patient, music for presentation to the patient, graphics and/orvideos (e.g., game for controlling a patient's breathing or breath-hold)for presentation to the patient, or any combination of the foregoing. Insome cases, if breathing game is presented to the patient, theprocessing unit 12 may also provide the same images of the game fordisplay on the screen 14 in synchronization with the images beingpresented to the patient. This way, the user of the apparatus 10 mayexplain the game to the patient in real time, and may visualize how thepatient reacts to the game.

In some embodiments, the processing unit 12 may also be configured todetermine an image of a patient, and output the image of the patient fordisplay on the screen based on the viewing direction of the user of theapparatus. By means of non-limiting examples, the image comprises a CTimage, a x-ray image, a MRI image, an ultrasound image, a tomosynthesisimage, an on-line image, a dose image, etc.

In one or more embodiments, the processing unit 12 may be configured toreceive a real-time consultation from a person who is different from theuser of the apparatus, and provide guidance information for display onthe screen for assisting the user to determine and/or to adapt atreatment plan. In one implementation, the apparatus 10 may include acommunication module configured to provide communication with one ormore persons outside the treatment room. Data obtained from within thetreatment room may be transmitted via the communication module to theperson(s) outside the treatment room. The person(s) outside thetreatment room may analyze the data, and provide input for reception bythe apparatus 10 in the treatment room. For example, the apparatus 10may detect a posture of the patient in the treatment room, and maytransmit the detected posture to a station outside the treatment room. Aperson at the station may analyze the posture to determine if it matchesthat prescribed in the treatment plan. If the posture does not match theprescribed posture, the person at the station may then send a message tothe apparatus 10 to inform the user of the apparatus 10 that thepatient's posture needs to be adjusted. The message may also indicatehow to adjust the patient.

In some embodiments, the processing unit 12 may be configured to obtaininformation from previous treatment sessions, and provide suchinformation for display on the screen 14 of the apparatus 10. The usermay utilize such information from previous treatment sessions todetermine whether to adjust a treatment plan to obtain a new treatmentplan for the current treatment session. For example, the processing unit12 may be configured to obtain multiple positions of an isocenter atdifferent respective times, and provide a graphic indicating change(s)of the isocenter over time for display on the screen. In anotherexample, the processing unit 12 may be configured to obtain multiplevalues of dose at different respective times, and provide a graphicindicating how the dose varies over time. In a further example, theprocessing unit 12 may be configured to provide patient setupinformation for display on the screen, the patient setup informationindicating weight change and/or positional change, of a patient. In oneimplementation, outlines of the patient (e.g., while supported by thepatient support) may be obtained at different times, and the weightchange may be indicated by changes to the outline of the patient.

In one or more embodiments, the apparatus 10 may further include adatabase configured to store data documenting one or more activitiesthat occur in a treatment room. The database may be wirelessly connectedto a part of the apparatus 10, or may be in communication with the partof the apparatus via a cable. In some cases, the database may be outsidethe apparatus 10, e.g., in the treatment room, or in another room suchas a treatment console. In one implementation, the database may beimplemented using cloud technology, wherein data from the apparatus 10may be sent to the “cloud”, and the apparatus 10 may obtain data fromthe “cloud”. In other cases, the database may be implemented using astorage device that is located within the apparatus 10. By means ofnon-limiting examples, the database may be configured to store datarepresenting a treatment setup configuration, a patient setupconfiguration, a patient behavior, or any combination of the foregoing.In another example, the database may be configured to store dataindicating how a treatment was executed. For example, the data mayindicate positions of a component of a treatment machine at differentrespective times, and/or a timing of energy delivery. In someembodiments, the database may have a database structure that isspecifically configured to store the data mentioned above. Such databasestructure is advantageous and is an improvement in the technology oftreatment analysis because it allows an executed treatment to be “playedback” so that the executed treatment can be retrospectively analyzed,and results of the treatment may be processed in association with theactivities of the executed treatment.

Also, in some embodiments, the apparatus 10 may include a camera unitcoupled to the processing unit 12. The camera unit may include anoptical camera, a depth camera, or both the optical camera and the depthcamera. The camera unit may be configured to capture various scenes inthe treatment room, and may output a corresponding video. In some cases,the video may be store in the database described previously. Also, insome cases, the video may be analyzed by the processing unit 12 toderive various information, such as patient identification, patientposition, patient posture, positions of various components of thetreatment system, etc.

FIG. 8 illustrates a method 800 in accordance with some embodiments. Themethod 800 may be performed by an apparatus, such as the apparatus 10,in a medical process. The medical process may be a treatment process.Also, the treatment process may be a radiation treatment process. Inother embodiments, the treatment process may involve other types ofenergy that is different from radiation. By means of non-limitingexamples, the treatment energy involved in the treatment process may beradiofrequency energy, thermal energy, ultrasound energy, proton energy,electron energy, etc. Also, in some embodiments, the medical process mayinvolve a particle accelerator. For example, the medical process may bea proton treatment process, a radiation therapy, a x-ray process, a CTimaging process, etc., each of which involves use of a particleaccelerator. Referring to the figure, the method 800 includes:obtaining, by a processing unit of the apparatus, treatment planinformation (item 802); obtaining, by the processing unit of theapparatus, a viewing direction of a user of the apparatus (item 804);processing, by the processing unit of the apparatus, the treatment planinformation based on the viewing direction of the user of the apparatusto create a graphical representation of the treatment plan informationfor presentation to the user (item 806); and displaying the graphicalrepresentation in a screen of the apparatus (item 708). In someembodiments, the processing unit involved in items 802, 804, 806 may bethe processing unit 12 of the apparatus 10. In other embodiments, theprocessing unit involved in items 802, 804, 806 may be any otherprocessing unit.

In some embodiments, the act of obtaining the treatment plan informationin item 802 may be performed by the processing unit receiving thetreatment plan information from another component. The treatment planinformation may be considered as an example of medical informationdiscussed previously (e.g., with reference to method 700). In otherembodiments, the act of obtaining the treatment plan information in item802 may be performed by the processing unit determining (e.g., deriving)the treatment plan information based on data received by the processingunit.

Also, in some embodiments, the act of obtaining the viewing direction ofthe user of the apparatus in item 804 may be performed by the processingunit receiving the viewing direction from another component, such as anorientation sensor. The orientation sensor may be coupled to the samedevice that contains the processing unit, or may be separate from theprocessing unit. In other embodiments, the act of obtaining the viewingdirection in item 804 may be performed by the processing unitdetermining the viewing direction based on data received by theprocessing unit.

In addition, in some embodiments, the act of processing the treatmentplan information based on the viewing direction of the user of theapparatus to create the graphical representation of the medicalinformation in item 806 may be performed by the a graphics generator ofa processing unit (e.g., the processing unit 12), which determines aconfiguration of the graphical representation based on the viewingdirection of the user. In some embodiments, the treatment planinformation comprises image data, such as CT data. In such cases, theprocessing unit creates a CT image rendering based on the viewingdirection, so that when the CT image rendering is displayed in a screen,the CT image rendering will correspond with a position of a patient asdirectly viewed by the user, or will correspond with a position of animage of the patient as viewed by the user. In some embodiments, theprocessing unit is configured to create the cross section of the CTimage along isocenter axes. In other embodiments, the processing unit isconfigured to create the cross section of the CT image along a directionthat is orthogonal to the viewing direction of the user.

In other embodiments, the treatment plan information may comprise adepth of a treatment isocenter. In such cases, the depth of thetreatment isocenter may be rendered by the processing unit over apatient (e.g., with respect to a viewing direction of the user of theapparatus 10), or for display in an overlay configuration with an image(e.g., a real-time image) of the patient.

In further embodiments, the treatment plan information may be any ofother information described herein. By means of non-limiting examples,the treatment plan information may be information regarding a desireddose, a target region, a critical organ, a desired patient posture, adesired patient position, positions of energy sources, trajectory of acomponent of a treatment machine, etc.

In some embodiments, the act of displaying the graphical representationin item 708 may be performed by a screen that is partially or completelytransparent. Such configuration allows a user of the apparatus todirectly view the real world. In other embodiments, the act ofdisplaying the graphical representation in item 808 may be performed bya screen that is a digital display of a virtual-reality device. In oneimplementation, the act of displaying the graphical representation maybe performed by the screen 14 of the apparatus 10.

Specialized Processing System

FIG. 9 is a block diagram illustrating an embodiment of a specializedprocessing system 1600 that can be used to implement various embodimentsdescribed herein. For example, the processing system 1600 may beconfigured to provide one, some, or all of the functions of theapparatus 10 in accordance with some embodiments. Also, in someembodiments, the processing system 1600 may be used to implement thegraphics generator 30, medical information module 20, room informationmodule 32, patient information module 22, and/or the viewing directionmodule 24. The processing system 1600 may also be an example of anyprocessor described herein.

Processing system 1600 includes a bus 1602 or other communicationmechanism for communicating information, and a processor 1604 coupledwith the bus 1602 for processing information. The processor system 1600also includes a main memory 1606, such as a random access memory (RAM)or other dynamic storage device, coupled to the bus 1602 for storinginformation and instructions to be executed by the processor 1604. Themain memory 1606 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by the processor 1604. The processor system 1600 furtherincludes a read only memory (ROM) 1608 or other static storage devicecoupled to the bus 1602 for storing static information and instructionsfor the processor 1604. A data storage device 1610, such as a magneticdisk or optical disk, is provided and coupled to the bus 1602 forstoring information and instructions.

The processor system 1600 may be coupled via the bus 1602 to a display167, such as a cathode ray tube (CRT), for displaying information to auser. An input device 1614, including alphanumeric and other keys, iscoupled to the bus 1602 for communicating information and commandselections to processor 1604. Another type of user input device iscursor control 1616, such as a mouse, a trackball, or cursor directionkeys for communicating direction information and command selections toprocessor 1604 and for controlling cursor movement on display 167. Thisinput device typically has two degrees of freedom in two axes, a firstaxis (e.g., x) and a second axis (e.g., y), that allows the device tospecify positions in a plane.

In some embodiments, the processor system 1600 can be used to performvarious functions described herein. According to some embodiments, suchuse is provided by processor system 1600 in response to processor 1604executing one or more sequences of one or more instructions contained inthe main memory 1606. Those skilled in the art will know how to preparesuch instructions based on the functions and methods described herein.Such instructions may be read into the main memory 1606 from anotherprocessor-readable medium, such as storage device 1610. Execution of thesequences of instructions contained in the main memory 1606 causes theprocessor 1604 to perform the process steps described herein. One ormore processors in a multi-processing arrangement may also be employedto execute the sequences of instructions contained in the main memory1606. In alternative embodiments, hard-wired circuitry may be used inplace of or in combination with software instructions to implement thevarious embodiments described herein. Thus, embodiments are not limitedto any specific combination of hardware circuitry and software.

The term “processor-readable medium” as used herein refers to any mediumthat participates in providing instructions to the processor 1604 forexecution. Such a medium may take many forms, including but not limitedto, non-volatile media, volatile media, and transmission media.Non-volatile media includes, for example, optical or magnetic disks,such as the storage device 1610. A non-volatile medium may be consideredan example of non-transitory medium. Volatile media includes dynamicmemory, such as the main memory 1606. A volatile medium may beconsidered an example of non-transitory medium. Transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise the bus 1602. Transmission media can also take theform of acoustic or light waves, such as those generated during radiowave and infrared data communications.

Common forms of processor-readable media include, for example, a floppydisk, a flexible disk, hard disk, magnetic tape, or any other magneticmedium, a CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, a RAM, a PROM, and EPROM,a FLASH-EPROM, any other memory chip or cartridge, a carrier wave asdescribed hereinafter, or any other medium from which a processor canread.

Various forms of processor-readable media may be involved in carryingone or more sequences of one or more instructions to the processor 1604for execution. For example, the instructions may initially be carried ona magnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to the processing system1600 can receive the data on the telephone line and use an infraredtransmitter to convert the data to an infrared signal. An infrareddetector coupled to the bus 1602 can receive the data carried in theinfrared signal and place the data on the bus 1602. The bus 1602 carriesthe data to the main memory 1606, from which the processor 1604retrieves and executes the instructions. The instructions received bythe main memory 1606 may optionally be stored on the storage device 1610either before or after execution by the processor 1604.

The processing system 1600 also includes a communication interface 1618coupled to the bus 1602. The communication interface 1618 provides atwo-way data communication coupling to a network link 1620 that isconnected to a local network 1622. For example, the communicationinterface 1618 may be an integrated services digital network (ISDN) cardor a modem to provide a data communication connection to a correspondingtype of telephone line. As another example, the communication interface1618 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, the communication interface1618 sends and receives electrical, electromagnetic or optical signalsthat carry data streams representing various types of information.

The network link 1620 typically provides data communication through oneor more networks to other devices. For example, the network link 1620may provide a connection through local network 1622 to a host computer1624 or to equipment 1626 such as a radiation beam source or a switchoperatively coupled to a radiation beam source. The data streamstransported over the network link 1620 can comprise electrical,electromagnetic or optical signals. The signals through the variousnetworks and the signals on the network link 1620 and through thecommunication interface 1618, which carry data to and from theprocessing system 1600, are exemplary forms of carrier wavestransporting the information. The processing system 1600 can sendmessages and receive data, including program code, through thenetwork(s), the network link 1620, and the communication interface 1618.

Although particular embodiments have been shown and described, it willbe understood that it is not intended to limit the claimed inventions tothe preferred embodiments, and it will be obvious to those skilled inthe art that various changes and modifications may be made withoutdepartment from the spirit and scope of the claimed inventions. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than restrictive sense. The claimed inventions areintended to cover alternatives, modifications, and equivalents.

1. An apparatus for use in a medical process that involves a particleaccelerator, comprising: a processing unit configured to obtaintreatment plan information, obtain a viewing direction of a user of theapparatus, and process the treatment plan information based on theviewing direction of the user of the apparatus to create a graphicalrepresentation of the treatment plan information for presentation to theuser of the apparatus; and a screen for displaying the graphicalrepresentation.
 2. The apparatus of claim 1, wherein the treatment planinformation comprises a position of an energy source for delivering atreatment beam.
 3. The apparatus of claim 2, wherein the graphicalrepresentation comprises a line representing a trajectory of thetreatment beam.
 4. The apparatus of claim 1, wherein the treatment planinformation comprises an expected configuration of a component of atreatment machine.
 5. The apparatus of claim 4, wherein the expectedconfiguration comprises an expected position of the component of thetreatment machine.
 6. The apparatus of claim 1, wherein the treatmentplan information comprises an expected dose for an internal target of apatient.
 7. The apparatus of claim 6, wherein the graphicalrepresentation indicates the expected dose graphically, and wherein theprocessing unit is configured to provide the graphical representationfor display over a patient as viewed through the display, or for displayin an overlay configuration with an image of the patient, so that thegraphical representation is at a location that corresponds with aposition of the internal target of the patient.
 8. The apparatus ofclaim 1, wherein the treatment plan information comprises an expectedposture of a patient.
 9. The apparatus of claim 1, wherein the treatmentplan information comprises a target position, a target size, a targetshape, a critical organ position, a critical organ size, a criticalorgan shape, or any combination of the foregoing.
 10. The apparatus ofclaim 1, wherein the treatment plan information comprises a targetfluence, and the processing unit is configured to provide the graphicalrepresentation for representing the target fluence.
 11. The apparatus ofclaim 1, wherein the treatment plan information comprises a trajectoryof a component of a treatment machine, and wherein the graphicalrepresentation is configured to indicate the trajectory of the componentof the treatment machine.
 12. The apparatus of claim 1, wherein theprocessing unit is configured to simulate a treatment based on thetreatment plan information, and wherein the graphical representationcomprises one or more images represented the simulated treatment. 13.The apparatus of claim 12, wherein the one or more images comprises asequence of images forming a video.
 14. The apparatus of claim 13,wherein each of the images in the video is based on a viewing directionand position of the user of the apparatus.
 15. The apparatus of claim12, wherein the simulated treatment comprises a simulated movement of acomponent of a treatment machine.
 16. The apparatus of claim 15, whereinthe graphical representation comprises a video showing the simulatedmovement of the component of the treatment machine.
 17. The apparatus ofclaim 1, further comprising a user interface for allowing the user todetermine a new treatment plan by selecting the new treatment plan froma plurality of pre-determined treatment plans, while a patient issupported on a patient support.
 18. The apparatus of claim 1, furthercomprising a user interface for allowing the user to determine a newtreatment plan by changing a parameter of a current treatment plan,while a patient is supported on a patient support.
 19. The apparatus ofclaim 1, further comprising a wearable device, wherein the screen is apart of the wearable device.
 20. The apparatus of claim 19, furthercomprising an orientation sensor coupled to the wearable device, whereinthe processing unit is configured to vary the graphical representationbased on an input from the orientation sensor.
 21. The apparatus ofclaim 19, further comprising a positioning device coupled to thewearable device, wherein the processing unit is configured to vary thegraphical representation based on an input from the positioning device.22. The apparatus of claim 19, wherein the wearable device comprises avirtual-reality device.
 23. The apparatus of claim 1, wherein the screencomprises a transparent screen for allowing the user to see surroundingspace.
 24. The apparatus of claim 1, wherein the screen is a part of ahandheld device.
 25. The apparatus of claim 1, wherein the graphicalrepresentation has a variable configuration that corresponds with theviewing direction of the user.
 26. The apparatus of claim 1, furthercomprising a camera unit coupled to the processing unit.
 27. Theapparatus of claim 26, wherein the camera unit comprises an opticalcamera, a depth camera, or both the optical camera and the depth camera.28. The apparatus of claim 1, wherein the processing unit is configuredto provide the graphical representation for display over a patient asviewed through the display, or for display in an overlay configurationwith an image of the patient.
 29. The apparatus of claim 1, wherein theprocessing unit is also configured to determine an image of a patient,and output the image of the patient for display on the screen based onthe viewing direction of the user of the apparatus.
 30. The apparatus ofclaim 25, wherein the image comprises a CT image, a x-ray image, a MRIimage, an ultrasound image, a tomosynthesis image, an on-line image, ora dose image.
 31. The apparatus of claim 1, wherein the processing unitis configured to determine a treatment dose, and output a graphicrepresenting the treatment dose for display on the screen based on theviewing direction of the user of the apparatus.
 32. The apparatus ofclaim 1, further comprising a user interface for allowing the user tocontrol a position of an energy source, a patient support, one or morecamera(s), one or more alignment laser(s), one or more light(s), acalibration of a device, a speaker for communication with a patient,music for presentation to the patient, or any combination of theforegoing.
 33. The apparatus of claim 1, wherein the processing unit isconfigured to receive a real-time consultation from a person who isdifferent from the user of the apparatus, and provide guidanceinformation for display on the screen for assisting the user todetermine and/or to adapt a treatment plan.
 34. The apparatus of claim1, wherein the processing unit is configured to obtain multiplepositions of an isocenter at different respective times, and provide agraphic indicating change(s) of the isocenter over time for display onthe screen.
 35. The apparatus of claim 1, wherein the processing unit isconfigured to obtain multiple values of dose at different respectivetimes, and provide a graphic indicating how the dose varies over time.36. The apparatus of claim 1, wherein the processing unit is configuredto provide patient setup information for display on the screen, thepatient setup information indicating weight change and/or positionalchange, of a patient.
 37. The apparatus of claim 1, wherein theprocessing unit is configured to provide information regarding fluencevirtualization for display on the screen.
 38. The apparatus of claim 1,further comprising a database configured to store data documenting oneor more activities that occur in a treatment room.
 39. The apparatus ofclaim 38, wherein the data represents a treatment setup configuration, apatient setup configuration, a patient behavior, or any combination ofthe foregoing.
 40. The apparatus of claim 38, wherein the data indicateshow a treatment was executed.
 41. The apparatus of claim 38, wherein thedata indicates positions of a component of a treatment machine atdifferent respective times, and/or a timing of energy delivery.
 42. Theapparatus of claim 1, wherein the screen comprises a transparent portionfor allowing the user to view a real world.
 43. The apparatus of claim1, wherein the screen is a part of a holographic device configured toproject three-dimensional images in a field of view of the user inreal-time.
 44. The apparatus of claim 1, wherein the treatment planinformation comprises a simulated dose effect on a target region and/orcritical organ.
 45. The apparatus of claim 1, wherein the processingunit is configured to simulate an execution of a treatment plan todetermine the simulated dose effect on the target region and/or criticalorgan.
 46. A method performed by an apparatus in a medical process thatinvolves a particle accelerator, comprising: obtaining, by a processingunit of the apparatus, treatment plan information; obtaining, by theprocessing unit of the apparatus, a viewing direction of a user of theapparatus; processing, by the processing unit of the apparatus, thetreatment plan information based on the viewing direction of the user ofthe apparatus to create a graphical representation of the treatment planinformation for presentation to the user; and displaying the graphicalrepresentation in a screen of the apparatus.